Images of the Russian Empire: Colorizing the Prokudin-Gorsky photo collection

Kenny Wang
SID: 3037341680

UC Berkeley

Overview

Sergei Mikhaylovich Prokudin-Gorsky (1863-1944) [Сергей Михайлович Прокудин-Горский] was a Russian chemist and photographer best known for his pioneering work in color photography. Obtaining the Tzar's special permission to travel across the vast Russian Empire and take color photographs of everything he saw, Prokudin-Gorsky amassed a vast collection of photographs. By recording three exposures of each scene onto a glass plate using a red, a green, and a blue filter, he was able to capture full color. Sadly, with the outbreak of the Russian Revolution, he was never able to realize his dreams of combining the images. However, after his death, many of his photographs came into the possession of the US Library of Congress, which has digitized his collections.

This project implements algorithmic colorization techniques to bring life to Prokudin-Gorsky's original images.

Process

The first iteration of my alignment algorithm was a brute-force search to align the red and green images to the blue, with up to 18 pixels of deviation in either dimension. I elected to use normalized cross-correlation (NCC) rather than Euclidean distance as my image matching metric. To avoid overfitting to the messy edges of the images, the area around the margins was excluded from the calculation. This first iteration was able to align the smaller .jpg input images.

To allow my code to handle the much-larger .tif images, I implemented an image pyramid optimization. This was implemented recursively: if an image's dimensions were less than 500 pixels, the original search would be used, but otherwise, the function would recursively align a half-scale version of the images. Multiplying the resulting offsets by two, it would then exhaustively search a small area around the target offset for the best alignment. The maximum allowable offset (and therefore the search area) decreases with increasing image size, due to the recursive subcalls becoming more and more precise.

I also attempted to crop the oddly-colored edges of the images, testing a method which would delete lines from the edges if they were too saturated with any particular color. However, I found that this had the side-effect of removing the sky from some images, so I decided to remove this change. Instead, I added a simple-but-effective crop on the images by chopping off areas where the aligned images do not overlap, noticeably improving the borders but not completely erasing the black borders on the left and right sides.

Edge Detection

Finally, I used the Canny edge detection algorithm to preprocess the red, green, and blue images before alignment, which helps avoid issues where images have differing contrast (especially prominent in emir.jpg). This tends to produce offsets close to, but slightly different from the original base NCC implementation. For most images, the difference is not easily visible, but edge detection substantially improves emir.jpg. However, a close inspection suggests that for several of the images, like three_generations.png and train.jpg, the quality was slightly decreased. Therefore I have generally chosen to display the base NCC result images, except for emir.jpg and self_portrait.jpg, which use edge detection.

Edge Detection	Basic NCC
emir.jpg
R[107, 40] G[49, 24]	R[0, 0] G[0, 0]

Results and Visualization

Here are the results of the algorithm, with the offset shown below each image.

Aligned + Cropped	Naive Stacking
church.jpg
R[58, -4] G[25, 4]	R[0, 0] G[0, 0]
emir.jpg (edge detection)
R[107, 40] G[49, 24]	R[0, 0] G[0, 0]
harvesters.jpg
R[124, 13] G[60, 17]	R[0, 0] G[0, 0]
icon.jpg
R[89, 23] G[41, 17]	R[0, 0] G[0, 0]
lady.jpg
R[117, 12] G[55, 8]	R[0, 0] G[0, 0]
melons.jpg
R[176, 13] G[81, 10]	R[0, 0] G[0, 0]
onion_church.jpg
R[108, 36] G[51, 27]	R[0, 0] G[0, 0]
sculpture.jpg
R[140, -27] G[33, -11]	R[0, 0] G[0, 0]
self_portrait.jpg (edge detection)
R[176, 37] G[79, 29]	R[0, 0] G[0, 0]
three_generations.jpg
R[111, 11] G[53, 14]	R[0, 0] G[0, 0]
train.jpg
R[87, 32] G[43, 6]	R[0, 0] G[0, 0]
cathedral.jpg
R[12, 3] G[5, 2]	R[0, 0] G[0, 0]
monastery.jpg
R[3, 2] G[-3, 2]	R[0, 0] G[0, 0]
tobolsk.jpg
R[6, 3] G[3, 3]	R[0, 0] G[0, 0]

Acknowledgements

This project is a course project for CS 180. A small portion of the code was provided by course staff. Website template is used with permission from Bill Zheng in the Fall 2023 iteration of the class.