Here's some photos taken from the AMC Metreon Theater in San Francisco, from CalHacks 11.0. I spent most of the hackathon finishing this project, though the vibes were pretty great.
Let's make panoramas!! Yay!!
2. Image Warping
The main step in creating our image mosaics is computing the homography that determines the transformation between the two images. With our pre- and post- transformation points as (x, y) and (x', y'), we have:
To find the values a, b, etc to build our matrix, we solve the following equation using least squares:
(Equation images from Aayush Gupta's Fall 2023 project website)
Warping image 1 to meet image 2, and image 2 to meet image 1, produces the following.
3. Image Rectification
Here's a picture of some pretty creatively decorated trash cans from CalHacks.
By getting the coordinates for the 4 corners of the trash sign and computing a homography between those corners and a rectangle, we can rectify the sign quite nicely (figure on the right).
4. Image Mosaics
To complete our final mosaic, we first need to generate a set of masks to help composite the images together. Here are masks representing the locations of data from only image 1, only image 2, their intersection, and their union.
Additionally, to make the transition between the images more natural, we have a linear seam Gaussian mask centered on the mean of the centroids of the two images.
Combining the two images gives us the following:
Here's a similar mosaic of photos of the familiar Soda Hall.
Project 4B: Feature Matching for Autostitching
In the second part of the project, we implement automatic interest point detection and matching for auto-stitching. Great, it saves me the trouble of manually clicking corresponding points!!
5: Interest Point Detection
First, we use Harris corner detection to detect potential points of interest that can be matched between the two images. Instead of simply taking the strongest Harris corners, we also implement Adaptive Non-Maximal Suppression to select good points that are spread out. Before applying ANMS, I also filter out potential corners with intensity less than 0.05, to avoid detecting superfluous points in the sky.
Here, observe an image with all 15496 Harris points, the image with only the top 2500 Harris points, and the image with 500 points chosen using ANMS.
6: Extracting Feature Descriptors
Once the potential points have been located, we then need to extract feature descriptors from them in order to perform matching between the two images. We accomplish this by extracting 8x8 patches from around each point, with a spacing of 5 pixels between points (so in reality, a 40x40 patch sampled at intervals 5 apart). I have chosen to sample the pixels with color, to make matching easier.
Here's some example features.
7: Matching Features
Now that we have features from both images, it's time to match them. Following the paper, we will use Lowe thresholding to determine what are good matches. Calculating the squared+summed differences between pairs of features, we detect matches where the distance to the 1st-nearest-neighbor is less than half the distance to the 2nd-nearest-neighbor (Lowe threshold 0.5).
This actually seems to match features quite convincingly. Here's some matching pairs of features.
8: Random Sample Consensus (RANSAC)
Now, for the final step, we must actually derive the relevant homography between the two images using the matched pairs of points. We accomplish this using Random Sample Consensus (RANSAC).
This works by repeatedly (1500 times) sampling 4 random matched point pairs, computing the homography between them, and seeing how many other matched pairs agree with the homography (within a distance threshold of 3). The largest set of agreeing points after this process is then used to generate a least-squares homography just like in Project 4A. The final results are quite good!
I also took the liberty to switch the blending algorithm from the simple Gaussian blur used in Project 4A to a two-layer Laplacian pyramid technique like the one used in Project 2.
9: Reflection
This was a really fun (but challenging) project, and it was quite interesting to learn how simple linear algebra could be used to stitch images together. I was very happy that I was able to follow along with and implement the MOPS paper, and I find the methods used make a lot of sense.
I feel that the automatic image alignment is very slightly worse than my painstakingly hand-annotated versions, but the MOPS images are still quite good and automatic.
Acknowledgements
This project is a course project for CS 180. Website template is used with permission from Bill Zheng in the Fall 2023 iteration of the class.
