Computer Vision Project

Computer Vision @ In this computer vision assignment, you will implement a zero-shot recognition system which resembles the system proposed in Christoph Lampert et al.'s [1] paper. (You don't need to read the paper to do this coursework.) All you need to know about this system is that it models the probability of a certain object (e.g. polar bear) being present in the image using the probabilities of being present for each of the attributes that a bear is known to have. For example, if we detect the attributes \white", \furry", \bulbous", \not lean", \not brown" etc. in the image, i.e. attributes that a polar bear is known to have, we can be fairly confident that there is a bear in the image. Hence, we can recognize a polar bear without ever having seen a polar bear, if (1) we know what attributes a polar bear has, and (2) we have classifiers trained for these attributes, using images from other object classes (i.e. other animals). Follow the steps below to implement zero-shot recognition. First, copy the Animals with Attributes dataset. The dataset includes 50 animal categories/classes, 85 attributes, and 30; 475 images. The dataset provides a 50x85 [login to view URL] which you should read into Matlab using M = load('[login to view URL]'); An entry (i, j)=1 in the matrix says that the i-th class has the j-th attribute (e.g. a bear is white), and an entry of (i, j)=0 says that the i-th class doesn't have the j-th attribute (e.g. a bear is not white). Three feature types are provided: cq-hist, which are color histograms, sift-hist, which is a SIFT bag-of-words histogram, and caffe, which are features extracted from a deep convolutional neural network. Pick any one feature type to use. For any feature type, there is one text file for every image, and files are organized by animal categories. Full Image Set in JPEG format is not directly downloadable for copyright reasons. The paper [1] splits the object classes (not images) into a training and test set, for purposes of zero-shot recognition. In this scenario, the training classes are animals that your system will see, i.e. ones whose images the system has access to. In contrast, the test set contains classes (animals) for which your system will never see example images. The 40 training classes are given in [login to view URL] and the 10 test classes are given in testclasses.txt. (Use [c1, c2] = textread('[login to view URL]', '%u %s'); to read in the class names. You can use the same function but with a different second argument to read in testclasses.txt.) At each time, we will assume that a query image can only be classied as belonging to one of the 10 unseen classes, so chance performance (randomly guessing the label) will be 10%.You will use all or a random sample of all images from the training classes (or rather, their feature descriptors) to train a classier for each of the 85 attributes. The predicate matrix mentioned above tells you which animals have which attributes. So if a bear is brown, you should assign the "brown=1" tag to all of its images. Similarly, if a dalmatian is not brown, you should assign the tag "brown=0" to all of its images. You will use the images tagged with "brown=1" as the positive data in your classier, and the images tagged with "brown=0" as the negative data, for the "brown" classier. Use the Matlab fitcsvm function to train the classifiers. Save the model output by each attribute classier as the j-th entry in a models cell array (initialized as models = cell(85, 1);) Note that if you sample data in such a way that you have either no positive or no negative data for some attribute classier, you'll get a classifier that only knows about one class, which is a problem. However, for every attribute, there are some classes that do and some that don't have the attribute. So you just have to make sure you sample data from all classes, when training your attribute classiers. You now have one classifier for each attribute.