INDRIYATI ATMOSUKARTO
CSE576 PROJECT 3 - FULLY AUTOMATED PANORAMIC MOSAIC STITCHING
WEDNESDAY, MAY 11, 2005

Objective
Extend the previous project to implement a fully automatic panoramic  stitching system similar to AutoStitch
by Matthew Brown.

Parameters
Majority I just used the same parameters over and over.
minmatch = 20
RANSAC iterations = 100
RANSACthresh = 10
distanceratiothresh = 0.6

Results
Note: I cropped the black portion out of the images cause it's a waste of space
You can also view the results on my webpage  http://www.cs.washington.edu/homes/indria/project/CSE576project3/index.html

1. Mountain test sequence
Without bundle adjustment With bundle adjustment

360 view
360 view
The final error after bundle adjustment and before bundle adjustment was totally the same so the bundle
adjustment didn't do much I think. I tried finding the difference between the two images by switching windows
but couldn't really find any movements at all.

2. MS Lobby test sequence
 I tried using different distance ratio threshold for this sequence (0.4 and 0.6). I couldn't seem to find a perfect
solution, cause it would align some parts but misalign others. Also for this I used 10 as the minMatch parameter.

Without bundle adjustment
0.4
 0.6

360 view
360 view

With bundle adjustment
0.4
 0.6

360 view
360 view
Difference between without and with bundle adjustment: well if you squint hard enough you will see differences in the ceiling lines, but it doesn't get rid of
all the misalignment

3. Palo Alto test sequence
This test sequence must be one of those mysterious test sequence because I tried different parameter combinations and even different image as the center image
but I would still get a misalignment for each and every single runs. It's not the same image but it will always have a misalignment. I gave up :)

Without bundle adjustment
Image 1
 Image 21
 Image 31

360 view
360 view
360 view

With bundle adjustment

360 view
I did bundle adjustment on the first picture, it basically just shifts things around a bit but doesn't really correct the major misalignment.

4. Piazza test sequence
Before I ran this test sequence I heard people saying that you will get error because the program can
only handle one kind of orientation for all the input images while piazza has some images which are landscape
while the rest are portrait. So I rotated the landscape images to be portrait but actually that didn't work for me.
So I used back the original test set and it works much better. The misalignment happens exactly at the two landscape
images so I guess if I changed my code to handle two different orientation of the images specifically it might have
done much better.

Without bundle adjustment With bundle adjustment

360 view
360 view

I was hoping that bundle adjustment would fix the misalignment of the two landscape images but no such
luck.

5. Outside level 6 - one end of the terrace (Taken with Shani and Anna)
I ran this set one one row at a time because I didn't want to wait like 30 minutes and end up getting
image graph not connected or whatever allocation vector error. So first row worked fine, adding second row worked
as well, but boy did it take time before I could get all three rows to stitch. I had to change the minMatch to a much smaller
number (from 20 to 5). I believe it's all the sky's fault!

Without bundle adjustment
Middle row
 Middle and bottom row
 Bottom. middle and top row

360 view
360 view
360 view

With bundle adjustment

360 view
The bundle adjustment tried to fix the clouds at the expense of messing up the rest. 

6. Outside level 6 - opposite end of the terrace (Taken with Shani and Anna)
Same thing as above, I ran the program incrementally one row at a time. Unfortunately for this sequence I couldn't
get all three rows to stitch so I am just showing the middle and bottom row. Trust me I really tried,
even put 3 as the min match but I kept getting the image graph not connected error.

Without bundle adjustment
Middle row
 Middle and bottom row

360 view
360 view

With bundle adjustment

360 view

Some slight shifts here and there. One thing you will notice is the ghosting effect of the camera box on the ledge, that's because
when we took the first row of pictures we didn't realize the box was there and thought that it wasn't in the pictures then when
we took the second row we thought it would look nicer if there wasn't a box so we moved it. I know the rule is "never touch the scene"
but it still did a good job aligning so all is well.

7. Softball stadium (Taken with Ravi)
Without bundle adjustment With bundle adjustment

360 view
360 view
I think the bundle adjustment kind of shifted things around for the seats but not much.

Discussion
The first problem I had was with blending. I literally cut and paste the blending code from my second project
to the skeleton code for project 3 and just ran the code. I had black lines appearing and obvious seams in between
images. Then I realized that's because now the images overlap in both  horizontal and vertical directions (from all
sides) so I followed Rick's suggestion of putting two weights for the feathering function (one along the width of the
image and the other along the height of the image) normalizing them so that the center gets a weight of at most 255.

The second problem was when I tested the mountain test sequence and it just couldn't align properly. Then on Wednesday
Ian sent the bug fix for the rotation and it aligned perfectly.

The third problem was when I was running the 360 panoramas, the images would align perfectly at some places but be rotated
at about 180 degrees in some other places. Took me three days to realize that nothing was wrong with my code, it was just because
svd was not doing a good job. So after talking to Susumu, Yongjoon and Ravi I made changes to the svd function and additional
checking. Quoting from YongJoon "svd 50% of the time gives good computation, 40% of the time gives wrong computation and 10% gives garbage".
So one check is in the svd function itself to check whether the S matrix is not singular and another check is in least square to check
whether svd of A is indeed equal to U*S*Vt. If not I would rechoose two inliers or re-do the whole alignment for that particular image pair.

Lastly I don't know whether my bundle adjustment is doing that well. I mean most of the time the error when it converges
ends up being the same as the initial error. However it does make some difference on the images, more like it will align
some part but ends up misaligning others.