ICLP: Test of Time Awards

Dear LPers,

As PC chairs of ICLP 2012, we were charged by the ALP EC with selecting the most influential paper of ICLP/ILPS 1992 (20 years backward) and the most influential paper of ICLP 2002 (10 years backward). The awards given were entitled (good suggestion from the ALP president) Test of time award.

We were left free to operate in an autonomous way. We therefore started setting some general rules that we believe these rules allows for a fair procedure and we would like to share to the whole community.

Since the focus is on the most influential papers we believe that a good help can come from bibliometrical information. Therefore we decided to use

• biblio-metric information for first screening, producing a short list, and
• to use our own personal feeling based on our roughly 20 years experience in a second stage (if necessary)

Given that this was the first edition of this award was given we also had a look to 1991, and 2001 papers.

The first filtering was made with google scholar (publish or perish software). MIT press/LNCS (old) proceedings do not emerge well in official databases as ISI or Scopus. For 20 years-old papers we soon realized that there are some problems (e.g. 0 citations for the ICLP paper, 85 citation for a research report with the very same name, 100 of citations for a journal version of the paper with the very same name, etc.) We kept track of that using two columns. No problems of this kind emerged for 10 years old papers.

In order to minimize errors looking for the paper with the precise name (symbols in titles are not always handled correctly by these databases) we looked for a super-set in the following way. Look for the papers list (using DBLP or your own copy of the printed proceedings). Then choose an author of the paper with a surname without accents, in case break a tie with the author with a less common surname, then look for all his papers in the interval of publication between, say, 1990 and 1996, then analyze by hand the report looking for the paper or other papers very close to it.

Basically we selected 20 years papers with at least (roughly) 150 citations (including, when applies, the citations of the RR with the same name and sometime of the close journal publication with the same content). We also used ISI and SCOPUS in this short list to have another hint. For 10 years papers, bibliometric data is more precise but  references were lower and therefore we considered those reaching roughly 60.

The algorithm can be easily reproduced (but the data changes daily).

We report here the short list of the 20 years (we have removed our personal papers, some of them were present, but in any case they were not at the level of the winner).

• Phan Minh Dung. Negations as Hypotheses: An Abductive Foundation for Logic Programming
• Kalyan Muthukumar, Manuel V. Hermenegildo. Combined Determination of Sharing and Freeness of Program Variables through Abstract Interpretation
• Pascal Van Hentenryck, Yves Deville. The Cardinality Operator: A New Logical Connective for Constraint Logic Programming
• Maurice Bruynooghe, Danny De Schreye, Bern Martens. A General Criterion for Avoiding Infinite Unfolding During Partial Deduction of Logic Programs
• Sverker Janson, Seif Haridi. Programming Paradigms of the Andorra Kernel Language
• David B. Kemp, Peter J. Stuckey Semantics of Logic Programs with Aggregates
• Gert Smolka. Records for Logic Programming
• Michael Gelfond, Vladimir Lifschitz. Representing Actions in Extended Logic Programming. JICSLP 1992:559-573
• Rachel Ben-Eliyahu, Rina Dechter. Propositional Semantics for Disjunctive Logic Programs

The winner was out of any doubt Gelfond-Lifschitz paper (bibliometric was so clear that any other consideration is useless). Vladimir Lifschitz was there receiving the awards and giving us a great talk on that paper and its evolutions.

For the 10 years paper the situation was more balanced. The short list is the following.

• Nicolas Beldiceanu, Mats Carlsson. Revisiting the Cardinality Operator and Introducing the Cardinality-Path Constraint Family.
• Marc Denecker, Nikolay Pelov, Maurice Bruynooghe. Ultimate Well-Founded and Stable Semantics for Logic Programs with Aggregates.
• Gianluigi Greco, Sergio Greco, Ester Zumpano. A Logic Programming Approach to the Integration, Repairing and Querying of Inconsistent Databases.
• Alessandra Russo, Rob Miller, Bashar Nuseibeh, Jeff Kramer. An Abductive Approach for Analysing Event-Based Requirements Specifications.
• Harald Ganzinger, David A. McAllester. Logical Algorithms.
• Francois Bry, Sebastian Schaffert. Towards a Declarative Query and Transformation Language for XML and Semistructured Data: Simulation Unification. ICLP 2002:255-270

The paper by Alessandra Russo et al. was also the best student paper in ICLP 2001. PC members chose very well. However, we decided that Bry-Schaffert paper has most influence in the broader set of computer science community. They built a bridge between logic programming and semantic web that is continuing in these days with joint summer schools and meetings.

We would like, anyway, to congratulate other authors as well. They all did great work.

The funny thing is that all this work was kept secret. We did it very in advance in order to invite the winners. The ALP president (Gopal) discovered the winners only few days before the meeting when he had to print the awards. We don’t know if secrecy is the best choice but we decided this way.

A final point: the test-of-time awards end up being two invited talks. We didn’t worry about having good speakers, but we got lucky.

Agostino Dovier and Vitor Santos Costa