Modify Google Galaxy Nexus for Near Infrared (NIR) Imaging
Modifying smartphone camera module for Near Infrared (NIR) imaging such as night vision and multispectral imaging applications
by Chuong Tran, James Le, Eigen Imaging Inc, Rev. 5/26/2013
Table of Contents
- Tools, Supplies, Warning
- Camera Module Basic Construction
- Remove camera module from the smartphone
- Camera Module Modification Procedure
- Before and after spectral response
- Blue sky test image
- Infrared images
- Detecting plants chlorophyll
- Night Vision Through 4x Riflescope
- Camouflage Net Countermeasure
- Applications Demo Videos
This guide is meant for advanced DIY so the procedure is gears more toward the top level disassembly instructions but all critical points for a successful modification will be pointed out clearly. The Samsung Google Galaxy Nexus camera module will be used as a reference for pointing out key features. Even though there are many different camera modules in the marketplace, however, they are all very similar in constructions. Therefore, the goal of this guide is to provide a sufficient basic knowledge that you can apply to most other models as well. The Eigen Imaging Android and iOS apps for Multi-Spectral Imaging and Analysis can be used with or without this modification although the software algorithms are designed to process images of the entire visible to near IR spectrum.
Hint: To minimize down time to the use of your smartphone, one suggestion is to order a spare camera module to make the modification since it is fairly inexpensive and much less risky as a learning process. After the modification, swap it in, and ready to use with little time down time.
Tools, Supplies, Warning
Camera Module Basic Construction
Before carrying the modification it is necessary to first understand the basic construction of a camera module. As shown in Figure 1, a camera module basically consists of two subassemblies. The first upper assembly typically contains a focus motor body that holds a lens and an IR cutoff filter. The second lower subassembly contains an imaging sensor bonded to a ceramic carrier, and a carrier PCB.
Figure 1. Basic Construction of Camera Module
Remove camera module from the smartphone
Before taking the smartphone apart, the near-range focus of the camera needs to be determined. For the Samsung Galaxy Nexus, the minimum close-up focus distance is about 2 inches. This will be slightly different for each brand of smartphone. This first step is important and will be retested after the modification by taking a picture of a printed page with tiny fonts. Find the minimum focus distance of your smartphone, take a picture, and save the image as well as the printed paper that was used for this baseline minimum focus distance test. See Figure 2. The saved image should be sharp and the text should be aligned parallel to the display screen. That way the expected field-of-view at this distance can be seen. Make note of the character at the left edge of the screen and the character at the right edge of the screen.
Figure 2, Document minimum focus distance before starting
Hint: there are lots of instructions on the internet regarding the tear down of the Samsung Galaxy Nexus smartphone so it will not be repeated here. As an example, ifixit has excellent procedures here. Warning: the camera module is ESD sensitive. Please take proper ESD precautions.
Camera Module Modification Procedure:
The camera module will be referred to as CM in the instruction.
- Now that the CM out of the way, the first important task is to protect the lens with low residue masking tape so it wouldn’t get scratched. Kapton tape is ideal for this purpose since it is strong enough to stay in place and very easy to remove and clean up because it has low residue. Cut the tape small enough to stick on the lens barrel top surface so that the four tiny notches at the edge are exposed (See Figure 3). These notches will be used to "zero" out the defocus once the IR filter is removed.
Figure 3. Protect lens with tape before starting the mod
Turn the CM over to view the back side. A black, protective tape should be visible vertically along the back side. If this tape comes off easily then remove it, but if adheres strongly to the PCB, leave it on. Use double-sided tape to carefully mount this surface to a flat metal-base holding fixture (See Figure 4).
Figure 4. Mount camera module to base holding fixture
Read this step a few times before attempting it. Use a low power heat gun (250W or less) to warm up the bonding adhesive interface that bonds the lower sensor PCB subassembly to the upper focus housing subassembly. Move the heat gun along the exposed edges as shown in Figure 5. Hold the nozzle about 2-3 inches away from this bonding interface to avoid damage the PCB or the focus motor housing. The goal is to warm it up just enough so that the upper and lower subassembly can be separated without melting away these components. Refer to the cross-sectional diagram in Figure 6.
Figure 5. Warm up along these 3 edges to soften the bonding adhesive
Figure 6. Upper subassembly flips away from lower subassembly
If the procedure up to this point was done properly there should be extra room at the top of the mounting base. Use this space to fully support the side of the flipped-up upper subassembly. The existing double side tape should be sufficient to keep it from moving about (See Figure 7). Be extra careful not to touch the lower subassembly while working on the next step. Note: Do not clean off the bonding adhesive. It will be reflowed in the later re-bonding step.
Figure 7. Additional support for upper subassembly
Look up from the bottom of the focus motor housing (upper subassembly) . A near-IR cutoff filter can be seen bonded to a plastic frame (See Figure 8).
Carefully use the heat gun to soften the adhesive around the edge of the filter. Just like before, the idea here is to warm it up just enough to remove the filter. It may take some trial and error so be patient. The adhesive will soften to a point where the filter can be removed with tweezers. Do not to force it. If the filter breaks then ALL of the debris must be removed.
- Once the filter is removed, use the tweezers to pick any debris that may be left behind in the upper subassembly. Use ESD safe compressed air to blow clean the upper subassembly. Do the same for the lower subassembly. A thorough cleaning of these two subassemblies is crucial before joining them back together. If available use an Anti-Static air ionizer at this step.
Read the next few steps a few times before attempting it since custom fixtures might be needed. To rejoin the two subassemblies, carefully flip down the upper subassembly to match up with the lower subassembly. The double-sided tape might need to be severed to free the upper subassembly. Do this step with great care to avoid any dusts falling onto the sensor surface of the lower subassembly.
Once the two subassemblies are mated properly, squeeze it firmly (but not too aggressively) and use the heat gun to reheat the bonding interface to reflow the bonding adhesive. There should be enough bonding strength to temporarily hold them in place with just a little heating. Promptly clamp it with a small vise. Apply a structural bonding adhesive to "permanently" set the bond.
Now might be a good time to take a quick break before putting the CM back into the phone. To reinstall the modified camera, just perform the removal steps above in reverse order. Before that is done however, the rear plate needs to be modified slightly to accommodate a changeable filter holder for use in future applications. This will also provide an easy means to "zero" out the focus shift offset in the upcoming steps.
Figure 9 shows the back of the rear plate that had been removed previously. The lens / flash combo cover is a thin metal plate which is held in place by a double-sided tape in addition to the four bent metal tabs. The red arrows show the tab locations. Remove the protective green tape, gently pry the tab up straight and gently push it out of rear plate. If part of tab breaks it is not the end of the world since the full length of the tab is not required for future use as a changeable filter holder.
Figure 9. Removal tabs for lens / flash cover plate
After the lens / flash cover plate is removed, it should look like Figure 10. Store the lens / flash combo plate for later modification to hold a filter of choice. Now the smartphone can be reassembled and a focus adjustment needs to be performed to "zero" out the focus offset.
Figure 10. Lens / flash combo cover removed
The removal of the near IR filter will cause the proper focus point to shift closer toward the lens so the image will not be focused on the imaging sensor. Even though the CM has auto focus, it does not have enough range of motion to compensate for this big focus offset. Therefore the next step is required to adjust the lens barrel to "zero" out this focus offset. The procedure is simple but needs to be repeated as necessary.
Read the caution notes in this step carefully before attempting the procedure. Refer to Figure 11, there are four anchor points that could be used to grip and turn the lens barrel to adjust focus. Use a right angle tweezers and place the two prongs at the blue or red markings. Choose the blue or red as a set so the lens barrel can turn smoothly. If a semitransparent tape is used to protect the lens surface then leave it on until a good focus is reached. After good focus is attained it can be removed. Keeping the protective tape on as long as possible prevents accidental scratching during the adjustment. Delrin tweezers don’t scratch as much as metal ones but since they flex more it is harder to turn the lens barrel. In either case, be patient and take care not to slip the tweezers tips onto the lens surface. This is a repetitive step in which the camera must be turned off each time to adjust the focus and turned back on to verify the focus.
Before turning the lens barrel put one drop of Methanol at each of the four notches shown. Let it soak for a minute to soften the lens barrel locking adhesive. Then position the tweezers as shown in Figure 12 and give it a full clockwise turning initially. To have more control place the smartphone screen on a flat table surface padded by a paper towel. Hold the tweezers steady and turn the smartphone instead (Note: turn in the opposite direction, counter clockwise instead of clockwise). The first turn will have some resistance since the thread locking adhesive is loosening but subsequent turning will require much less effort.
Turn on the camera and let it focus at some distance object, if it is still out of focus, then turn the lens barrel again but only a quarter-turn at time. Verify it again with the camera turned on. Repeat this process until a decent focus is obtained for far distance objects (ie. an object that is at least 20 feet away). Turn counterclockwise or clockwise as needed to get good focus position. If the protective Kapton tape is left on, the difference in focus quality can still be seen even though it might not look sharp. Once in the ball park the tape should be removed for fine focus adjustments.
After adjustments are made for focusing of distant objects the near focus needs to be adjusted. Set up as shown in Figure 2, like the beginning of the baseline minimum focus test. Refer to the saved image earlier and try to get the same field of view to be in focus for this test.
Repeat the same refocusing procedure as the previous step for the near focus test. Hint: the near distant image should be somewhat in focus by now if the distant focus was achieved earlier.
If the camera is intended for use only for visible- to near-IR imaging then just put the lens/flash combo plate back in its place. This plate will be stay in place using just the four little protruding tabs. But once the decorative thin plastic plate cover is placed back on, everything will be more secured in place. This lens/flash combo plate can be easily modified to accept special filters. The procedure for that will be posted soon.
If dust spots appear on the image, don't panic, just blow the dust out with the clean compressed air. However the lens must be removed completely by turning the lens barrel counter-clockwise. Then point the compress air nozzle at about 2-3 inches from the imaging sensor to blow out any dust particles trapped in there. This is a repetitive process so patience is a virtue.
- Congratulation on a job well done!
Download the EigenCAM app, available for Android and iOS (coming soon), and have fun with your multispectral imaging enabled smartphone.
But Wait, There's More:
Blue sky test image
To confirm that there's no dust on the sensor, shoot an image of a clear blue sky like that of Figure 14. Any noticeable specks of dusk will reveal themselves clearly. If any dust spots are seen then go back to the above instructions on how to remove them. Best is to shoot the portion of a blue sky without any clouds in it. Keep the camera in the shade while shooting to avoid lens glare.
By populating the lens / flash cover plate with a different bandpass filter(s), a variety of interesting infrared images can be taken as shown in Figure 15. The top-left filter uses a special triple-band filter that blocks blue and passes narrow bands of green, red, and NIR. Top-middle uses a special triple-band filter that blocks green but passes narrow bands of blue, red, and NIR. Top-right uses a filter that passes wavelengths above 700nm. Bottom left uses RGB filter for standard color. Bottom middle also uses RGB filter but passes more blue. Bottom right shows the mounted filter in the lens /flash cover plate. Click on the image to enlarge it.
Camouflage Net Countermeasure