The CLEF Initiative (Conference and Labs of the Evaluation Forum, formerly known as Cross-Language Evaluation Forum) is a self-organized body whose main mission is to promote research, innovation, and development of information access systems with an emphasis on multilingual and multimodal information with various levels of structure. CLEF promotes research and development by providing an infrastructure for:
- multilingual and multimodal system testing, tuning and evaluation;
- investigation of the use of unstructured, semi-structured, highly-structured, and semantically enriched data in information access;
- creation of reusable test collections for benchmarking;
- exploration of new evaluation methodologies and innovative ways of using experimental data;
- discussion of results, comparison of approaches, exchange of ideas, and transfer of knowledge.
The CLEF Initiative is structured in two main parts:
- a series of Evaluation Labs, i.e. laboratories to conduct evaluation of information access systems and workshops to discuss and pilot innovative evaluation activities;
a peer-reviewed Conference on a broad range of issues, including
- investigation continuing the activities of the Evaluation Labs;
- experiments using multilingual and multimodal data; in particular, but not only, data resulting from CLEF activities;
- research in evaluation methodologies and challenges.
Since 2000 the CLEF has played a leading role in stimulating investigation and research in a wide range of key areas in the information retrieval domain, becoming well-known in the international IR community. It has also promoted the study and implementation of appropriate evaluation methodologies for diverse types of tasks and media. Over the years, a wide, strong, and multidisciplinary research community has been built, which covers and spans the different areas of expertise needed to deal with the spread of CLEF activities.
The results were traditionally presented and discussed at annual workshops in conjunction with the European Conference for Digital Libraries (ECDL), now called Theory and Practice on Digital Libraries (TPDL).
Since 2010, CLEF has taken the form of an independent event, constituted by a peer-reviewed conference jointly organised with a set of evaluation labs.