WK 01: Prep and Test Scans
Welcome back to my blog! I I just recently graduated this past spring and while I am still job searching I’ve been helping my professor get his VFX Lab started. The lab is currently working on a new project and I am excited for this project because we will be working with the The Crow Museum of Asian Art of The University of Texas at Dallas and digitize many of their artifacts. The plan is to take photos and align them using Reality Scan in order to create accurate 3D models of these museum pieces. Currently I am still learning about what the best methods would be for taking photos, but we have a couple of test scans we’ve done so far this week. Below I’ll explain the information that I have learned so far, the test scans that were taken, and my plans moving forward!
What I’ve Learned So Far
There are so many useful and free resources online that have helped me greatly! Reality Scan has a lot of documentation and the program itself even guides you through the all of the steps. Most of what I will be going over in this section you can find on the epic games website, so if you want a more detailed breakdown feel free to visit their website. Here is a quick gist of the outline of things I’ve learned:
Scanning Basics
When scanning objects it is important to get sufficient coverage and have 70% overlap. The computer uses color information from the pixels in order to align the images together, which means its important that the photos are clear and in focus. When scanning an object you need to consider the size, surface material, subject accessibility, and the lighting in the environment. These factors can change how you approach scanning the subject. The three core principles of photogrammetry are image quality, information overlap, and subject coverage.
Photography
I have barely used a professional camera before and so I had a lot of learning to do for this part. First, the basic things to consider is good image resolution, no image processing, and a high-resolution camera that can capture in RAW file format. It’s important to have the correct exposure and lighting, ideally there should be flat lighting with little shadows and highlights, this is known as the diffuse/flat lighting technique.
A deep depth of field tends to be preferable, which means a higher f-Stop. Smaller aperture or less light coming in the camera also helps when taking images.
The Exposure Triangle
In the Photogrammetry Basics by Quixel, they also go over how aperture, shutter speed, and ISO are the three most important things to consider when setting the right exposure for your images. This chart from the Quixel course was very useful in understanding how these 3 settings work.
4. Color Checker
The color checker helps tell the software what the accurate of the subject is. When using a color checker, you can set it next to the object for the first image taken, then remove it for the rest of the scan. It needs to be in the same lighting as the subject. This can be used in pre-processing with software such as Adobe Lightroom in order to correct the color values. The color checker can be adjusted according to the correct known values, and then you can apply a color correction pattern to the rest of the images from that scan.
5. Mind Mapping and Filling the Frame
When it comes to more complex objects it is best to map out sections of it to scan. For example, when scanning a statue you can section off the legs, arms, and head.
It’s best to try to fill the frame with object, but also the entire object does not need to be visible in every frame. This will help the computer with registering the pixels in the image. Its best to avoid capturing unnecessary detail in the background.
6. Coverage
It is very important for the captures to be at consistent intervals apart from each other. The intervals can be no more than 30 degrees apart. It is also important that the images are in a sort of chain, at a consistent distance from each other, so it is best to aim for shooting complete loops around the subject.
For large objects, like buildings, it is important to get multiple passes. Then, it is good to get an “over-all” pass from farther away, capturing the entire subject in the frame, or at least larger chunks of it. The same rule applies where you need to ensure to get 70% overlap and coverage all around. Finally, you can then do a more detailed pass by shooting closer to the subject to get the finer details. This data will be especially helpful for creating textures.
7. Flash
It is good to use flashes and polarization in order to try to capture the subject in more neutral lighting. Sometimes the environment surrounding the subject can cast harsh lights or strong shadows, for example warm light from the sun, that will affect how the models look when trying to use them in projects that require a different 3D environment.
8. Scanning on a Turntable
It is more ideal to have the turntable move in increments and stop rather than move in a continuous motion, as to avoid any motion blur.
You can put markers on the turntable, and this will help the program align the images.
9. Control Points
During the test scans the images wouldn’t always align properly so I was curious if there was a method I could use to manually align them. Obviously this method isn’t preferred and it would be ideal to have good quality photos that will align properly, but I figured learning this would be helpful to have as a crutch. As it turns out, you can create control points in Reality Scan to tell it where to reference a certain point among the images. There are two methods that I saw through tutorials online that you can go about setting the control points. One is by going through the images and clicking on spot that will be easy for the computer to recognize. The only problem with this is that it is a pretty tedious method. The second is by using the 3D viewport after hitting preview on the mesh model tab, and hovering your mouse over the model to see which point on the model reaches the most amount of cameras. This method is faster, although still a bit more time consuming than I would like.
Test Scans
This week we were able to take multiple passes of photos for different objects! There was one object in particular that was giving me the most trouble though, which was the piggy bank. We also took photos of a tissue box which yielded much better results. I’ll elaborate more on both of them below:
The Nefarious Piggy Bank
By this point I have taken so so many pictures of this wicked piggy bank. It definitely presents me with an interesting challenge. I’ve tried taking photos of it with 30 degree and 15 degree increments going up in loops around the piggy bank, then flipped it upside down and repeated the process. Next I take the object into Reality Scan and create masks for each side separately, and then finally put all the images and masks together to be aligned. For some reason it had a hard time aligning all of the images together. Typically the images will align well if I only use the set of images from one side of the object, but I can’t get a full model because it refuse to align all the images, basically it won’t align the bottom and top properly to each other. We tried to resolve the problem by placing stickers around the piggy bank because we thought the issue was that there was too little information on parts of it for the computer to register the pixels. However, after placing the stickers on it, the computer seems to have an even more difficult time aligning the images.
At this point the issue could be multiple things. The first culprit I’ll have to deal with is how I scan it on the turntable. The camera is on autofocus and the turntable turns continuously in a circle without stopping, so I suspect that there isn’t enough focus on the images to get a clear scan and potentially some motion blur. We also don’t have all of our lighting equipment in yet so we have been relying on a ring flash mounted to the camera, but the ISO is set very high right now since the room overall isn’t lit well. I’m still trying to think of other possible solutions.
The Tissue Box
The tissue box is a simple shape and has a distinguishable pattern so it was able to scan pretty easily thankfully. I repeated the same process of taking loops of images around the box going up in 15 degree increments for the top and bottom. I then created the masks and finally aligned all the images, which most were able to align successfully.
Next Steps
There are a couple of things I would like to do such as:
Ensure that the f-stop and aperture settings look good.
Use the color checker, especially with the piggy bank and see if this helps.
Adjust the turntable and have it stop in increments instead of letting it move continuously.
Do some more research on what could be affecting the images for the piggy bank so much besides the reasons already listed.
If I still have no luck with the piggy bank tomorrow it may be best for me to wait until we get the rest of our lights in to retake the loop of pictures and move on to a different object for now. Don’t worry though! I am determined to see that piggy bank model on my computer screen and will continue to troubleshoot different solutions! Thanks for reading, and I’ll see you next week!
