This page contains some useful information about the Information Retrieval project.

In the project, you will have to develop a Relatedness function, i.e. a function that, given two entities, returns their semantic relatedness. Entities are expressed as Wikipedia pages. For example:

R(Tree, Tree) = 1.0
R(Tree, Leaf) = 0.9
R(Tree, Dart_Vader) = 0.0

Note that TagMe is built upon such a function.

For assisting your development, you are provided with:

• An access to ferrax-2.itc.unipi.it
• A helper class which provides you with a few methods that you may find useful
• An experiment class that launches two experiments for you
• A site that keeps track of the development for each group.

We suggest you perform the following steps to get into the environment.

1. Ask x@y (where x is 'cornolti' and y is 'di.unipi.it') for a username and a password to access ferrax-2.
2. Log into ferrax-2 with:

   ssh username@ferrax-2.itc.unipi.it

3. Have a look at the jar libraries that you may need to use:

   ls /home/irproject/*lib*

4. Note the presence of ir2013project.jar. This jar provides you with two classes:

   irproject.IRProjectHelper
   irproject.IRProjectExperiments

   IRProjectHelper gives you access to the Wikipedia graph, while IRProjectExperiments runs two experiments to test your function.

5. Check the available ram and the hard disk usage. You have to keep an eye at these values: if they run too low, problems may occur

   free -h
   df -h

As our first relatedness function, we will make a dummy one that always returns 0.

1. Open DummyRel.java with your favorite text editor.
2. Copy and paste the following code:

   DummyRel.java

   import it.acubelab.tagme.RelatednessMeasure;
    
   public class DummyRel extends RelatednessMeasure {
    
           public DummyRel(String lang) {
                   super(lang);
           }
    
           @Override
           public float rel(int a, int b) {
                   return 0.0f;
           }
    
   }

   Then save and exit.

3. Make a directory called bin/ to store your compiled files, set an environment variable for the path of the libraries, and compile:

   mkdir bin
   export IRLIB=../irproject/lib/*:../irproject/tagme_libs/*:../irproject/tagme_preproc_lib/*
   javac -cp $IRLIB -d bin/ *.java

4. Cool. You have made your first relatedness function. Let's test it. Create a Main.java file and paste the following code:

   Main.java

   import it.acubelab.tagme.config.TagmeConfig;
   import it.acubelab.tagme.RelatednessMeasure;
   import irproject.IRProjectExperiments;
    
   public class Main {
           public static void main(String[] args) throws Exception {
                   TagmeConfig.init("/home/irproject/config.xml");
                   String groupName = ""; // Insert your group name here.
                   String groupPw = ""; // Insert your group password here.
                   RelatednessMeasure rel = new DummyRel("en");
                   IRProjectExperiments.launchTagMeExperiment(groupName, groupPw, rel);
                   IRProjectExperiments.launchRelatednessExperiment(groupName, groupPw, rel);
           }
   }

5. Before testing your relatedness function, let's have a look at the scoreboard page. This page shows the achievements of the other groups. It also shows the baseline given by TagMe.
6. We are ready to launch. Enter:

   java -cp $IRLIB:bin Main

   On the first launch, the program will have to query Wikipedia and retrieve some data. Don't worry: this data gets cached, and if you run the program again, the output will be way smaller. Running the program again will generate the following output:

   Results for the Evaluation of TagMe:
   F1:0.524198
   Precision:0.577957
   Recall:0.479590
   Time spent for the annotations:20.935064
   
   Results for the Evaluation of the Relatedness function:
   Quadratic distance:0.369874
   Absolute distance:0.408762

   Look back at the scoreboard page. Your group should have appeared.

Let's have a closer look at these numbers. There are two kinds of results:

• The evaluation of the performance of TagMe using your relatedness function, and
• The evaluation of the function itself.

You don't really have to care about how the first three numbers are computed (if interested, read here), but keep in mind that the higher those numbers are, the better TagMe is performing. Let's focus on the last two figures. The test of the relatedness function is performed against a dataset of 311 pairs (entity, entity). For each pair, the dataset provides a relatedness found by humans. Your task is to make a relatedness function that computes a relatedness as close as possible to that given by those humans.
Let P be the list of pairs, H(p) the relatedness found by the humans for a pair of entities p, and R(p) the relatedness found by your function for p. The measures we employ are defined as:

• Absolute distance:
• Quadratic distance:

In other words, they compute the distance between your and their relatedness, and do an average. Quadratic distance penalizes bigger mistakes more than Absolute distance.

public static int[] getInlinks(int page_id);
public static int[] getOutlinks(int page_id);
public static int TitleToId(String title);
public static String getCategoryTitle(int catId);
public static IntSet getAllWids();
public static boolean isDisambiguation(int pageId);
public static boolean isNormalPage(int page_id)
public static boolean isPerson(int pageId);
public static int[] getCategories(int pageId);
public static int dereference(int pageId);
public static float linkProbability(string anchor);
public static float commonness(string anchor, int pageId);