Modify Google Galaxy Nexus for Near Infrared (NIR) Imaging

Modifying smartphone camera module for Near Infrared (NIR) imaging such as night vision and multispectral imaging applications

by Chuong Tran, James Le, Eigen Imaging Inc, Rev. 5/26/2013  

 

Table of Contents

  1. Introduction
  2. Tools, Supplies, Warning
  3. Camera Module Basic Construction
  4. Remove camera module from the smartphone
  5. Camera Module Modification Procedure
  6. Before and after spectral response
  7. Blue sky test image
  8. Infrared images
  9. Detecting plants chlorophyll
  10. Night Vision Through 4x Riflescope
  11. Camouflage Net Countermeasure
  12. Applications Demo Videos

 

Introduction

This guide is meant for advanced DIY so the procedure is gears more toward the top level disassembly instructions but all critical points for a successful modification will be pointed out clearly.  The Samsung Google Galaxy Nexus camera module will be used as a reference for pointing out key features.  Even though there are many different camera modules in the marketplace, however, they are all very similar in constructions.  Therefore, the goal of this guide is to provide a sufficient basic knowledge that you can apply to most other models as well.  The Eigen Imaging Android and iOS apps for Multi-Spectral Imaging and Analysis can be used with or without this modification although the software algorithms are designed to process images of the entire visible to near IR spectrum.  

Hint: To minimize down time to the use of your smartphone, one suggestion is to order a spare camera module to make the modification since it is fairly inexpensive and much less risky as a learning process.  After the modification, swap it in, and ready to use with little time down time.

 

Tools, Supplies, Warning

  • A low power 10x microscope with a ring light.  If not available then a large field-of-view magnifying glass can be used.
  • Kapton tape to protect the lens since it is easier to clean up but standard masking tape is ok.
  • A right angle tweezers to adjust lens focusing.  Metal tweezers are better but care must be taken not to scratch the optics.
  • Double sided tape to mount the camera module to a small metal block as holding fixture.
  • A sharp razor blade or a X-acto knife to slice through the double sided tape.
  • Clean, particle-free compressed air to blow dust off the sensor.  ESD safe is ideal.
  • Structural adhesive for resealing. High temperature type is ideal.
  • A small clamp to hold the camera module while curing the structural adhesive.
  • A few drops of high quality Methanol.   Electronics grade is preferred but lesser grade would also work.
  • Lots of patience and a clean surface with good illumination.
  • PERFORM AT YOUR OWN RISK!

  • Camera Module Basic Construction

    Before carrying the modification it is necessary to first understand the basic construction of a camera module.  As shown in Figure 1, a camera module basically consists of two subassemblies.  The first upper assembly typically contains a focus motor body that holds a lens and an IR cutoff filter.  The second lower subassembly contains an imaging sensor bonded to a ceramic carrier, and a carrier PCB.

     

    Figure 1.  Basic Construction of Camera Module

     

    Remove camera module from the smartphone

    Before taking the smartphone apart, the near-range focus of the camera needs to be determined.  For the Samsung Galaxy Nexus, the minimum close-up focus distance is about 2 inches.  This will be slightly different for each brand of smartphone. This first step is important and will be retested after the modification by taking a picture of a printed page with tiny fonts.  Find the minimum focus distance of your smartphone, take a picture, and save the image as well as the printed paper that was used for this baseline minimum focus distance test.  See Figure 2.  The saved image should be sharp and the text should be aligned parallel to the display screen.  That way the expected field-of-view at this distance can be seen.  Make note of the character at the left edge of the screen and the character at the right edge of the screen.

     

     

    Figure 2,  Document minimum focus distance before starting

     

    Hint: there are lots of instructions on the internet regarding the tear down of the Samsung Galaxy Nexus smartphone so it will not be repeated here.   As an example, ifixit has excellent procedures here.  Warning: the camera module is ESD sensitive. Please take proper ESD precautions.

     

    Camera Module Modification Procedure:

    The camera module will be referred to as CM in the instruction.

    1. Now that the CM out of the way, the first important task is to protect the lens with low residue masking tape so it wouldn’t get scratched.  Kapton tape is ideal for this purpose since it is strong enough to stay in place and very easy to remove and clean up because it has low residue.  Cut the tape small enough to stick on the lens barrel top surface so that the four tiny notches at the edge are exposed (See Figure 3).  These notches will be used to "zero" out the defocus once the IR filter is removed.

      Figure 3. Protect lens with tape before starting the mod

       

      1. Turn the CM over to view the back side.  A black, protective tape should be visible vertically along the back side.  If this tape comes off easily then remove it, but if adheres strongly to the PCB, leave it on.  Use double-sided tape to carefully mount this surface to a flat metal-base holding fixture (See Figure 4).

      Figure 4.  Mount camera module to base holding fixture

       

      1. Read this step a few times before attempting it.  Use a low power heat gun (250W or less) to warm up the bonding adhesive interface that bonds the lower sensor PCB subassembly to the upper focus housing subassembly.  Move the heat gun along the exposed edges as shown in Figure 5.  Hold the nozzle about 2-3 inches away from this bonding interface to avoid damage the PCB or the focus motor housing.  The goal is to warm it up just enough so that the upper and lower subassembly can be separated without melting away these components.  Refer to the cross-sectional diagram in Figure 6.

         


        Figure 5.  Warm up along these 3 edges to soften the bonding adhesive

         

         


        Figure 6.  Upper subassembly flips away from lower subassembly

          

        1. If the procedure up to this point was done properly there should be extra room at the top of the mounting base. Use this space to fully support the side of the flipped-up upper subassembly. The existing double side tape should be sufficient to keep it from moving about (See Figure 7).  Be extra careful not to touch the lower subassembly while working on the next step.  Note: Do not clean off the bonding adhesive.  It will be reflowed in the later re-bonding step.

           

          Figure 7.  Additional support for upper subassembly

           

        2. Look up from the bottom of the focus motor housing (upper subassembly) .  A near-IR cutoff filter can be seen bonded to a plastic frame (See Figure 8).

        3. Carefully use the heat gun to soften the adhesive around the edge of the filter.  Just like before, the idea here is to warm it up just enough to remove the filter.  It may take some trial and error so be patient.  The adhesive will soften to a point where the filter can be removed with tweezers.  Do not to force it.  If the filter breaks then ALL of the debris must be removed.


            Figure 8. Remove near IR cutoff filter

             
             
            1. Once the filter is removed, use the tweezers to pick any debris that may be left behind in the upper subassembly.  Use ESD safe compressed air to blow clean the upper subassembly.  Do the same for the lower subassembly.  A thorough cleaning of these two subassemblies is crucial before joining them back together.  If available use an Anti-Static air ionizer at this step.
            1. Read the next few steps a few times before attempting it since custom fixtures might be needed.  To rejoin the two subassemblies, carefully flip down the upper subassembly to match up with the lower subassembly.  The double-sided tape might need to be severed to free the upper subassembly.  Do this step with great care to avoid any dusts falling onto the sensor surface of the lower subassembly.

            1. Once the two subassemblies are mated properly, squeeze it firmly (but not too aggressively) and use the heat gun to reheat the bonding interface to reflow the bonding adhesive.  There should be enough bonding strength to temporarily hold them in place with just a little heating.  Promptly clamp it with a small vise. Apply a structural bonding adhesive to "permanently" set the bond.

            1. Now might be a good time to take a quick break before putting the CM back into the phone.  To reinstall the modified camera, just perform the removal steps above in reverse order.  Before that is done however, the rear plate needs to be modified slightly to accommodate a changeable filter holder for use in future applications. This will also provide an easy means to "zero" out the focus shift offset in the upcoming steps.

            1. Figure 9 shows the back of the rear plate that had been removed previously.  The lens / flash combo cover is a thin metal plate which is held in place by a double-sided tape in addition to the four bent metal tabs.  The red arrows show the tab locations.  Remove the protective green tape, gently pry the tab up straight and gently push it out of rear plate.  If part of tab breaks it is not the end of the world since the full length of the tab is not required for future use as a changeable filter holder.

              Figure 9.  Removal tabs for lens / flash  cover plate

               

              1. After the lens / flash cover plate is removed, it should look like Figure 10. Store the lens / flash combo plate for later modification to hold a filter of choice.  Now the smartphone can be reassembled and a focus adjustment needs to be performed to "zero" out the focus offset.




              Figure 10.  Lens / flash combo cover removed

               
               
              1. The removal of the near IR filter will cause the proper focus point to shift closer toward the lens so the image will not be focused on the imaging sensor.  Even though the CM has auto focus, it does not have enough range of motion to compensate for this big focus offset. Therefore the next step is required to adjust the lens barrel to "zero" out this focus offset.  The procedure is simple but needs to be repeated as necessary.

              1. Read the caution notes in this step carefully before attempting the procedure.  Refer to Figure 11, there are four anchor points that could be used to grip and turn the lens barrel to adjust focus.  Use a right angle tweezers and place the two prongs at the blue or red markings.  Choose the blue or red as a set so the lens barrel can turn smoothly.  If a semitransparent tape is used to protect the lens surface then leave it on until a good focus is reached.  After good focus is attained it can be removed.  Keeping the protective tape on as long as possible prevents accidental scratching during the adjustment.  Delrin tweezers don’t scratch as much as metal ones but since they flex more it is harder to turn the lens barrel.  In either case, be patient and take care not to slip the tweezers tips onto the lens surface.  This is a repetitive step in which the camera must be turned off each time to adjust the focus and turned back on to verify the focus.

              Figure 11. Anchor points for adjusting focus
               
               
               
              1. Before turning the lens barrel put one drop of Methanol at each of the four notches shown.  Let it soak for a minute to soften the lens barrel locking adhesive.  Then position the tweezers as shown in Figure 12 and give it a full clockwise turning initially.  To have more control place the smartphone screen on a flat table surface padded by a paper towel.  Hold the tweezers steady and turn the smartphone instead (Note: turn in the opposite direction, counter clockwise instead of clockwise). The first turn will have some resistance since the thread locking adhesive is loosening but subsequent turning will require much less effort.




                Figure 12.  Adjusting focus after camera module mod
                   
                   
                  1. Turn on the camera and let it focus at some distance object, if it is still out of focus, then turn the lens barrel again but only a quarter-turn at time.  Verify it again with the camera turned on.  Repeat this process until a decent focus is obtained for far distance objects (ie. an object that is at least 20 feet away).  Turn counterclockwise or clockwise as needed to get good focus position.  If the protective Kapton tape is left on, the difference in focus quality can still be seen even though it might not look sharp.  Once in the ball park the tape should be removed for fine focus adjustments.

                  1. After adjustments are made for focusing of distant objects the near focus needs to be adjusted.  Set up as shown in Figure 2, like the beginning of the baseline minimum focus test.  Refer to the saved image earlier and try to get the same field of view to be in focus for this test.

                  1. Repeat the same refocusing procedure as the previous step for the near focus test.  Hint: the near distant image should be somewhat in focus by now if the distant focus was achieved earlier.

                  1. If the camera is intended for use only for visible- to near-IR imaging then just put the lens/flash combo plate back in its place.  This plate will be stay in place using just the four little protruding tabs.  But once the decorative thin plastic plate cover is placed back on, everything will be more secured in place.  This lens/flash combo plate can be easily modified to accept special filters.  The procedure for that will be posted soon.

                  1. If dust spots appear on the image, don't panic, just blow the dust out with the clean compressed air.  However the lens must be removed completely by turning the lens barrel counter-clockwise.  Then point the compress air nozzle at about 2-3 inches from the imaging sensor to blow out any dust particles trapped in there.  This is a repetitive process so patience is a virtue.

                  1. Congratulation on a job well done! 
                  1. Download the EigenCAM app, available for Android and iOS (coming soon), and have fun with your multispectral imaging enabled smartphone.

                  But Wait, There's More:

                   
                  Before and after spectral response
                   
                  The montage of images in Figure 13 below show expected data and results.  The top left shows the break out of the camera module in which the internal filter needs to be removed.  The top-middle image shows RGB response of the stock filter while the top-right shows visible-to-near-IR response of the sensor after the stock filter is removed as measured by the Ocean Optics USB4000-VIS-NIR spectrometer.  The bottom row proves that there are no change in form, fit, or function to the smartphone after modification.  Click on the image to enlarge the photo.


                  Before vs After Spectral Response

                  Figure 13.  Before and after spectral response
                   
                   
                   

                  Blue sky test image

                  To confirm that there's no dust on the sensor, shoot an image of a clear blue sky like that of Figure 14.  Any noticeable specks of dusk will reveal themselves clearly.  If any dust spots are seen then go back to the above instructions on how to remove them.   Best is to shoot the portion of a blue sky without any clouds in it.  Keep the camera in the shade while shooting to avoid lens glare.

                  Blue Sky Test Image
                   

                  Figure 14.  Blue sky test image
                   

                  Infrared images

                  By populating the lens / flash cover plate with a different bandpass filter(s), a variety of interesting infrared images can be taken as shown in Figure 15.  The top-left filter uses a special triple-band filter that blocks blue and passes narrow bands of green, red, and NIR.  Top-middle uses a special triple-band filter that blocks green but passes narrow bands of blue, red, and NIR. Top-right uses a filter that passes wavelengths above 700nm.  Bottom left uses RGB filter for standard color.  Bottom middle also uses RGB filter but passes more blue.  Bottom right shows the mounted filter in the lens /flash cover plate.  Click on the image to enlarge it.



                  Changeable front filters
                  Figure 15.  RGB vs Infrared images
                   
                   
                   
                   
                   
                  Detecting plants chlorophyll
                   
                  Figure 16 shows a practical use of multispectral imaging for detecting which plants in the natural environment have higher moisture content and which ones are under stress.  These minute effects can be seen through multispectral imaging before the human eye can detect these changes. The image below uses a filter that transmits 400-650nm and a narrow band around 850nm.  The plants in darker red are healthier due to their higher moisture content.  Getting plenty of water allow them to grow better than those shown as lighter pink.  A few plants in the back in lighter yellow, due to the lack of chlorophyll, are not getting enough water which may wither and die if not taken care of soon. 
                  Figure 16.  Detecting plants chlorophyll
                  Night Vision Through 4x Riflescope
                   
                  Figure 17 shows the utility of using the modified smartphone on a 4x scope to assist night surveillance. The modified Galaxy Nexus smartphone was mounted to a DIY mount adapted to fit on the Pentax 4x riflescope.  The building was over 1.5 miles driving distance as seen on Google map and it was lit by surrounding street lights.  The attached high quality IR Streamlight flashlight works well at about 85 meters (279 ft) range.
                   
                  Galaxy Nexus on Riflescope
                   
                  Night Vision Galaxy Nexus Modified
                  Figure 17.  Night Vision Through 4x Riflescope

                   

                  Camouflage Net Countermeasure

                  Figure 18 shows another utility of using the modified smartphone on a 4x scope to detect camouflage netting.  The range for this test was about 85 meters (279 ft) away.  The picture on the left shows that the camo net can be easily seen without much effort with the enhancement algorithm ON.  In normal camera viewing as shown in the right picture, one could not quickly discriminate the camo net that was placed over the trees.  Click on image to enlarge for more details.
                   
                  Camouflage countermeasure
                   
                  Figure 18.  Camouflage Countermeasure Through 4x Riflescope