Final Projects: Image Quilting and Facial Keypoint Detection using Neural Networks

Final Project 1: Image Quilting

Overview

Image quilting is a relatively straightforward algorithm for texture synthesis and texture transfer, originally introduced by Efros and Freeman in their SIGGRAPH 2001 paper. Texture synthesis involves generating a larger texture image from a small sample by intelligently combining overlapping patches to ensure seamless transitions. Texture transfer extends this idea by giving an object the appearance of a texture sample while preserving its underlying shape, such as overlaying a texture onto a face while maintaining facial features. The Efros-Freeman algorithm works by sampling patches from the texture and arranging them in overlapping patterns, minimizing visual discrepancies in the overlapping regions. This is achieved by computing optimal seams, where pixels with similar intensities are selected for smooth blending. Here optimal is in the sense of SSD between two overlapping images and the seam is found using min-cut.

Results

Final Project 2: Facial Keypoint Detection using Neural Networks

Overview

In this project, we automate the detection of facial keypoints using convolutional neural networks (CNNs) in PyTorch, building on previous work from Project 3 that involved manual keypoint selection. Facial keypoints, such as those around the eyes, nose, and mouth, are identified on faces in images to enable advanced analysis. In the case of Project 3, we used keypoints for morphing. We use the IMM Face Database and the IBUG face in the wild dataset. This project progressively develops a robust neural net-based system capable of detecting faces and predicting the precise locations of facial keypoints.

Results