Problem Definition:

The ability to combine multiple images has many applications in the field of graphics such as the creation of enviroment maps for games and special effects. Images however must be transformed using non affine transformations in order to make them align properly.

Method & Implementation:

The method used was based on the paper by Richard Szeliski, "Video mosaics for virtual environments". The following is a pseudocode of the project:

Initialization
BaseImg=LoadImage(Img(1))

I=2
When Add Button Click
{

LoadImage(Img(I))
Define Tie Points Betwen BaseImg and Img(I)
M=Calculate AffineMatrix that warps Img(I) onto BaseImg using initial tie points
Iterations=1
While(Error<ErrorThreshold) or MaxIterations has been reached

{
Render Img(I) into BaseImg Coordinates using M
For each Pixel(i,j) calculate partials derivitives of the error with respect to M
For each Pixel(i,j) use img(I) to calculate x,y image gradients
For each Pixel(i,j) calculate error Img(I)((i,j) - BaseImg(i,j)
Use these values to calculate matrices A,B as described in paper
Use matrices A,B to calcuate deltaM as described in paper
Calculate error between Img(I) and BaseImg using M+deltaM to warp img(i)
Recalculate deltaM by incrementing sigma if error has not decreased
Iterations++
}

Create a Mask of all points where Img(1)(in BaseImg coordinates) and BaseImg overlap
Find the minLeft Value of the Mask and the maxRight value of the mask
Create Mask2 where all columns < minLeft =[0,0..], all colums > maxRight =[1,1...1] and all columns in between
range from [0 0 .. 0] to [1,1,...1] based on their position. i.e. [1,1,..1] * (j-minLeft)/(maxRight - minLeft) where j is the column index.
If(Img(I)) overlaps to the right of BaseImg then invert the Mask2, ie Mask2=Identity-Mask2
BaseImg=Mask2 .* BaseImg + (Identity- Mask2).*Img(I) (.* is element by element multiply)

}

Here is a zip file of the source code.

Description of the important modules:
gui.m - Gui Functions, handles buttons, and manages data

initM.m - initialize M using the tie points

insideTest.m -test if a point(X,Y) is inside the boundaries defined by the dimentions of an image

regenBase.m - merges the current base image with the warped image to produce a new base image
very similar to render but a bit faster since it just averages the images

render.m - render 2 images togheter using a weighted function at the overlap

sqrdError - returns a value which is the sum of the errors in intensities between two grey scale images

updateM2 - Gradient Descent Functions based off Video Mosaics for Virtual Enviroments

Experimental Results:

Conclusion:

The gradient method to align images works well but my usefullness is limited because of its dependence on tie points for intial estimates, it would be better if the program automatically aligned the images. Also a better blending function would be useless, I tried to implement a multiresolution image splining method but unfortunetly ran out of time before I finished.