Egocentric Activity Recognition for Cognitive Aid

We work on the recognition of Activities to support people with disabilities.

CVPR 2018 in Salt Lake City, Utah, USA

If you are at CVPR, come to have a look to our presentation on Wednesday at around 2:50-4:30 in the Ballroom.
Also, we will be happy to see you during the poster session on Wednesday from 4:30 to 6:30 in the Halls C-E. Our paper ID is 3516.
You can find more info here:

• Our paper: Egocentric Activity Recognition on a Budget
• Our newly created Dataset: DataEgo
• Our poster for Wednesday focused on the RL framework: Poster
• Our poster for Friday focused on the Dataset: Poster
• A video summarizing the activities in our dataset: Video

Duplication detection in Vines (short videos)

We described a set of videos and indexed them using a fractal curve which let us represent each keyframe containing temporal information using just 32 bytes.

Then we look for the links among groups of keyframes to detect duplications. We worked with opencv, c++ and python on a team conformed by 5 people.

See a project presentation here

CBIR Visualization

Master Project: I worked on the visualization of CBIR systems.

First, it was proposed a novel approach named Clustering Set for the interactive search that displays the result set in a flexible manner. It is able to group elements with similar content using clustering techniques and to organize the result set in a well-defined tree with radial combinations. The advantage of using the clustering and radial combination is to combine positive aspects of different visualization strategies. It was used over images and videos.

Then, I modeled and implemented an flexible framework that supports the dynamic creation of several hybrid visual designs, based on the combination of different visualization strategies following the Clustering Set idea ( This framework also use clustering techniques to find coherent relationships among the data distributing the results in successive radial layouts).

Furthermore, we can deal with several levels of clusters which allow us to form fractal structures.

Finally, new visualization structures are created in a straightforward manner using such framework as you can see in the images bellow. Please, click in any image if you want to expand it.

This Figure shows you a fractal structure of three combined techniques: Spiral, Concentric Rings and Circular layouts (base of the Clustering Set)




Also, you can see a short video showing the implemented framework:


Some documents about this project are:

• Journal paper entitled "Navigating Through Video Stories Using Clustering Sets" can be found here.
• Technical report from those in the Institute of Computing at Unicamp can be found here.
• Poster Presentation in the Women in Machine Learning Workshop can be seen here.
• A paper entitled "FISIR: A Flexible Framework for Interactive Search in Image Retrieval Systems" has been accepted for the 21st International Conference on MultiMedia Modelling MMM. The paper can be seen here.

Image Registration for Faces

The project was developed over a collection of faces from people, 10 face images for each person.
The purpose of the project was to detect who was the person in the new face image. So several small transformation (rotation, translation, scale) where made on the query image according to the gradient descend technique to register it with the known patterns (person) from the collection and evaluate which person fit better.

Clustering Scientific texts

We took all the papers from the ACM from 2003 to 2007 (75 171 archives). Then we processed the abstract and obtained representative words for each paper by using text processing techniques. Finally, we grouped the words in clusters of words to find representative words for an area of knowledge.

I worked in a team conformed by 3 people. More details about this project can be found here.

Interactive image segmentation

We worked on the semisupervised segmentation of images considering several classifiers.

I worked on a team conformed by 8 people and on a subteam on neural networks conformed by 3 people.

A paper about this topic was published in the Sibgrapi conference, as you can see here.

Kohonen Neural network graphical simulator

I work on the design and implementation an interactive tool to show how the kohonen network works. Several options were implemented to show how the parameters affect the network behavior in real time.

The users select and model a pattern and start the simulation. At the beggining, the neurons from our network do not know the given pattern (cyan square). At the end, the Kohonen network learns the pattern and cover the pattern surface.
This process can be appreciated in the Figures bellow in four gradual stages A, B C and D.

A: --- B:
C: --- D:
A short showing a trained network of the implemented kohonen can be downloaded downloaded or watched

A prediction system for dairy cattle Production

This project had the purpose to predict dairy cattle production based on fuzzy neural networks. 5 characteristics from each animal (cow) were the inputs of a the neural network.