Object Detection Using the OpenCV / cvBlobsLib Libraries

A short example of how to utilize various open source library functions that can be used to identify and analyse strongly connected components for a given input image.

In the example I have given here, the image represents microarray sample spots printed to a slide using a Xaar inket printer.  Using our robotic equipment, a camera is mounted to the printhead, so that images are taken of the spots, as they are being printed on-the-fly, usually in linear groups of 12 or 32 at a time:

ObjectDetection1

As part of an investigation into how we may improve our quality control (QC) processes, one task (of many) will be to analyse such an input image, checking the spot images for things like misalignment in the x,y axes, spot shape (circularity), missing spots or tiny spots (satellites). It is anticipated that this would greatly speed up QC, which at present rely on manual validation.

In reality, the initial spot images will be subject to a degree of perspective distortion, given that the camera is mounted at an angle of approximately 30 degrees to the perpendicular of the slides.  Some corrective matrix transformations would be needed to make the image rectangular, as opposed to quadrilateral.  Another time.

For blob extraction, I have used the CvBlobsLib, a library to perform connected component labelling on binary images, available at the OpenCVblobslib github page.  It is reasonably simple to use. I use the OpenCV libraries to open a colour image, convert it to a grayscale and then thresholding to convert it to a black and white (binary) image:

ObjectDetection2

I then use the filter method in CBlobResult to extract all blobs in the image that conform to a certain size, count the number of ‘proper’ blobs discovered and display them in red:

ObjectDetection3

C++ example code is here:

#include "stdafx.h"
#include "BlobResult.h"
#include <cv.h>
#include <cxcore.h>
#include <highgui.h>

const std::string filepath = "spots.bmp";

int _tmain(int argc, _TCHAR* argv[])
{ 

   CBlobResult blobs;
   CBlob *currentBlob;

   // Load grayscale version of coloured input image
   IplImage* original = cvLoadImage( filepath.c_str(),
                                     CV_LOAD_IMAGE_GRAYSCALE );

   // Make sure image file is available
   assert ( original );

   // Obtain binary (black and white) version of input image
   IplImage* img_bw = cvCreateImage( cvGetSize( original ),
                                     IPL_DEPTH_8U,
                                     1 );

   // Threshold to convert image into binary (B&W)
   cvThreshold( original,     // source image
                img_bw,       // destination image
                100,          // threhold val.
                255,          // max. val
                CV_THRESH_BINARY ); // binary type ); 

   // Find the white blobs in the B&W image
   blobs = CBlobResult( img_bw, NULL, 0 );

   // Exclude all white blobs smaller than the given value (80)
   // The bigger the last parameter, the bigger the blobs need
   // to be for inclusion
   blobs.Filter( blobs,
                 B_EXCLUDE,
                 CBlobGetArea(),
                 B_LESS,
                 80 );

   // Get the number of blobs discovered
   int num_blobs = blobs.GetNumBlobs(); 

   // Display the filtered blobs
   IplImage* filtered = cvCreateImage( cvGetSize( img_bw ),
                                       IPL_DEPTH_8U,
                                       3 ); 

   cvMerge( img_bw, img_bw, img_bw, NULL, filtered );

   for ( int i = 0; i < num_blobs; i++ )
   {
      currentBlob = blobs.GetBlob( i );
      currentBlob->FillBlob( filtered, CV_RGB(255,0,0));
   }

   // Display the input / output windows and images
   cvNamedWindow( "input" );
   cvNamedWindow( "output" );
   cvShowImage("input", img_bw );
   cvShowImage("output", filtered);

   // Wait for user key press and then tidy up
   cvWaitKey(0);

   cvReleaseImage( &original );
   cvReleaseImage( &filtered );

   cvDestroyWindow( "input" );
   cvDestroyWindow( "output" );    

   return 0;
}

Sample image “spots.jpg” available here.

To make this actually build and work you first need to ensure the following things are in place:

Install OpenCV

Make sure the OpenCV libraries have been properly installed in your Visual Studio environment. Make sure this works all right before proceeding further.

Download and build the cvBlobsLib library

Download and extract the cvBlobsLib library placing the extracted folder in a suitable location.

After downloading and extracting this cvBlobsLib Visual Studio project, you then build it, so that the necessary cvblobslib.lib file gets created, either within a Debug or Release folder. Any new project you are working on that uses the cvBlobsLib library will need the cvblobslib.lib file in order to correctly work.

On building this for the first time, you will probably encounter compiler errors like these:

c:\dump\cvblobslib_opencv_v8_3\blobcontour.h(6): fatal error C1083: Cannot open include file: 'cv.h': No such file or directory
1> BlobResult.cpp
1>c:\dump\cvblobslib_opencv_v8_3\blobresult.h(24): fatal error C1083: Cannot open include file: 'cxcore.h': No such file or directory
1> BlobOperators.cpp

Notice that this project comes with the original project settings which would need to be changed:

As with any other project that uses OpenCV, the cvblobslib VC++ project will also need to be set up so that OpenCV is correctly installed, and it knows where to find the library files, additional includes etc.

See the same OpenCV posting for details, which includes a section for Visual Studio 2010 considerations.

If you still have trouble building and getting the necessary cvblobslib.lib file, here’s one I made earlier.

Set the Visual Studio Project Properties

1. If you’re not using a Windows console application as in my example but are using an empty project instead, then omit the “#include stdafx.h” bit. “stdafx” is created with either a new Win32 Console Project or a Console Application (.NET) application. (Thanks hiperchelo)

2. Make sure the cvblobslib.lib file you created earlier is copied into the project folder of the application you are working on.

3. In Project -> Properties -> C/C++ -> Additional Include Directories, add the location of the folder where the cvBlobsLib library was installed. If this isn’t done, the compiler will complain it can’t find the “BlobResult.h” header file.

4. In Project -> Properties -> Linker -> Input, add the cvblobslib.lib entry, in addition to any existing OpenCV library files.

If you’re using OpenCV2.1 for example, you will also need to include the {cv210, cvaux210, highgui210, cxcore210, etc}.lib files.  For OpenCV1.x versions, these will be {cv, cvaux, highgui, cxcore, etc}.lib files.

5. In Project -> Properties -> C/C++ -> Pre-Compiled Headers, select Not use precompiled headers.

6. In Project -> Properties -> C/C++ -> Code generation -> Run-time library, select Debug Multithreaded DLL (for debug version) or Multithreaded DLL (for release version).

7. In Project -> Properties -> General -> Use of MFC, select Use MFC in a shared DLL.

Build the Project

You may get a linker error during building your project that uses cvBlobsLib, similar to this garbage:

error LNK2019: unresolved external symbol "public: virtual __thiscall CBlobResult::~CBlobResult(void)" (??1CBlobResult@@UAE@XZ) referenced in function "public: struct _IplImage * __thiscall OpenCV_Handler::FindConnectedComponents(struct _IplImage *,class ConnComponentSet *,int const &,int const &)" (?FindConnectedComponents@OpenCV_Handler@@QAEPAU_IplImage@@PAU2@PAVConnComponentSet@@ABH2@Z)

If this is the case, then Visual Studio does not yet know about the cvblobslib.lib library file, against which it should link.  In Project Properties -> Linker -> Input -> Additional Dependencies check that this file has been included:

ObjectDetection4

 
During compilation you might get the following error message:

fatal error C1083: Cannot open include file: 'BlobResult.h': No such file or directory

If this is the case then make sure you have specified the necessary include file for using the cvBlobsLib libraries. In the Project Properties -> C/C++ -> General -> Additional Include Directories, ensure that the path to the includes has been added:

Other Issues: access violation errors when running under Release Mode

I noticed that using CBlobResult objects in Visual Studio’s Release Mode can cause access violation errors similar to the one shown:

ObjectDetection5

This can be corrected by making sure that the proper version of the cvblobslib.lib file is being used – it needs to be the one built under Release mode, not Debug Mode:

Open the CvBlobsLib Visual Studio project and do a clean and rebuild under Release Mode. Grab hold of the newly built cvblobslib.lib file contained in the Release folder and copy it into your project that is using the cvBlobsLib library. In other words, replace the old (probably Debug Mode) cvblobslib.lib file you have been using with the new Release Mode one.

Update: 22 July 2011

The library does the job for real-world instances too.  See this posting for tips on how to integrate the OpenCV/cvBlobsLib with the FlyCapture camera, by Point Gray Research.  The input image used was a sample subset of microarray spots printed using a Xaar inkjet printer approximately 150 microns in diameter, printed to a 75.0 x 25.0 mm glass slide, with black background, camera approximately 10 degrees to the perpendicular:

ObjectDetection6

Which is then converted into a grayscale image…

ObjectDetection7

And converted into a black and white binary image using the thresholding…

ObjectDetection8

And finally filtered, discarding blobs that are less than a given area, ignore the single-pixel blobs in white, this seems to be a display issue. The important thing is that the correct number and location of acceptable blobs were identified:

ObjectDetection9

See here for related posts.

If you are having problems compiling the original cvBlobsLib project to obtain the necessary .lib file, a copy is available here.

Download example Visual Studio 2010 Projects – 32/64 bit

If you are experiencing difficulty getting this up and running (I know what a pain this can be to set up), then some example Visual Studio 2010 projects are available for download that contain all the necessary configuration settings, lib/dll files etc. Projects are available for both x86 (32-bit) and x64 (64-bit) processors:

OpenCVproject

Everything is self contained within the download, using relative path settings so that you don’t have to worry about getting everything set up correctly according to your machine – just extract the 7zip file to a location of your choice, open the Visual Studio 2010 solution as per your processor type (x86 or x64) and start experimenting with OpenCV / cvBlobsLib libraries straight away.

Because of the size of the complete OpenCV library, only the lib/dlls necessary for running these examples are included in the download, but feel free to expand on these.

As always, comments and suggestions are always welcome. Feel free to contact me if you have any questions.

USD 2.49 / Download


Other posts related to image detection

Tracking Coloured Objects in Video using OpenCV
Displaying AVI Video using OpenCV
Analyzing FlyCapture2 Images obtained from Flea2 Cameras
Integrating the FlyCapture SDK for use with OpenCV
OpenCV Detection of Dark Objects Against Light Backgrounds
Getting Started with OpenCV in Visual Studio

29 Comments

  1. Hi! cvBloblib is a very good solution for blob tracking as I know. Now I try to build the binary libraries with MSVS 2008 but it compiles with many errors. How can I overcome this version incompatibility?

  2. Hi
    Very nice tutorial but i got error when i try to build .lib file.
    i am using visual studio 2010, have proper install OpenCv. Cmake well perform his job but when i try to build .lib file through Visual studio it gives different error for the different versions of CvBlobslib:
    1. for one version it said that it cant access some of openCv files so cant build.
    2. for second it build .dll file instead of static .lib
    3.for third version it say no proper files provide..
    in past i have already use blob liabrary with V C++6.0..
    but now here…Can you help me more plz
    during searching on internet i came to know that other programmers are also facing this problem..and only CvBlobslib Linux version is working..
    and whats your opinion .lib build on one Machine can be used on another or not??
    if()
    then plz share that with us

    Thanks

    • Hi Hamad you are using VS 2010. I see there is a difference for setting Visual Studio settings in VS 2010 and the older versions of VS. I will update the posting for setting up OpenCV shortly:

      http://www.technical-recipes.com/2011/setting-up-opencv-in-visual-studio/

      In the section “Configure Visual Studio” it says to go to Tools -> Options -> Projects and Solutions -> VC++ Directories etc. In Visual Studio 2010, this is no longer done globally as with older versions of VS, but on a per-project basis. Notice when you try to do this in VS 2010 it says that ‘Options has been deprecated’

      Instead, in your VS C++ project you need to go to project -> properties -> VC++ Directories and set the sections for

      – Include Directories
      – Library Directories
      – Source Directories

      Just give it the same information as given in the OpenCV set up posting. Have you done this yet?

      • Thanks Andy
        For your help..!
        i solved the issue by follow your instructions..
        but now i have raised a new problem..

        Can you understand any thing from this error (not error a type of assert() but its not bad for a single image but if i am runing through a video then its become worst)

        OpenCV Error: Incorrect size of input array (Specified element size doesn’t match to the size of the specified element type (try to use 0 for element type)) in unknown function, file ..\..\..\OpenCV SVN\modules\core\src\datastructs.cpp, line 388

        Plz direct me to get rid of from this error.. i am unable to run my finalized code..thaks in advance

  3. Error 4 error C1083: Cannot open include file: ‘cxcore.h': No such file or directory c:\program files\bloblib\cvblob\blobresult.h 24 1 cvblobslib

    Error 7 error C1083: Cannot open include file: ‘cxcore.h': No such file or directory c:\program files\bloblib\cvblob\blob.h 21 1 cvblobslib

    Error 6 error C1083: Cannot open include file: ‘cv.h': No such file or directory c:\program files\bloblib\cvblob\blobcontour.h 6 1 cvblobslib

  4. I downloaded the latest cvblobslib. I tried to load the cvblobslib.dsw in Visual c++ 2010 Express and it says it needs to convert it, but then it says it can’t load the cvblobslib.dsp and then to remove it. At the end it opens an empty project.

    So how can I compile this library with Vc2010 Express?

    If this is not possible with Express, could someone post the compiled cvblobslib.lib?

  5. I’m testing and I found another problem with the VC 2010 Express:
    it doesn’t find the afx.h includes (in blobresult.cpp):
    #include //suport per a CStrings
    #include //suport per a AfxMessageBox

    So it must commented out and also other two points in the cpp where there are Cstring.

    After that all works well.

  6. I inserted in the VC2010 project, all .cpp and .h files contained in the zip file.
    It gives me error in file blobproperties.cpp at line 36:
    t_contourList::iterator itContour;
    saying t_contourList isn’t a Class.
    if I remove that file and related .h, all works. But that file should be useful, getting Area and perimeter.

    What could it be?

  7. I have a problem when build the project.
    1>—— Build started: Project: SolDetec, Configuration: Debug Win32 ——
    1>cvblobslib.lib(blob.obj) : error LNK2038: mismatch detected for ‘_MSC_VER': value ‘1600’ doesn’t match value ‘1700’ in SolDetec.obj
    1>cvblobslib.lib(BlobResult.obj) : error LNK2038: mismatch detected for ‘_MSC_VER': value ‘1600’ doesn’t match value ‘1700’ in SolDetec.obj
    1>cvblobslib.lib(BlobContour.obj) : error LNK2038: mismatch detected for ‘_MSC_VER': value ‘1600’ doesn’t match value ‘1700’ in SolDetec.obj
    1>cvblobslib.lib(ComponentLabeling.obj) : error LNK2038: mismatch detected for ‘_MSC_VER': value ‘1600’ doesn’t match value ‘1700’ in SolDetec.obj
    1>cvblobslib.lib(BlobOperators.obj) : error LNK2038: mismatch detected for ‘_MSC_VER': value ‘1600’ doesn’t match value ‘1700’ in SolDetec.obj
    1>LINK : fatal error LNK1104: cannot open file ‘mfc100d.lib’
    ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

  8. It’s been a long time but I still have a problem here. Every time I try to debug cvblobslib.lib I have notification like this:
    “unable to start program ‘C:\cvblobs\.\Debug\cvblobslib.lib’
    The specified file is an unrecognized or unsupportted binary format”
    Can you help me?
    Thanks

  9. Thanks a ton for the tutorial, Andy. I’m a university student working on an image detection project (for which the Hough circles algorithm wasn’t quite cutting it), and this information was unbelievably helpful.

    I think I’ve followed your directions correctly, and while I’ve configured Visual C++ 2010 to find the .lib file you posted, when I try to write out some of the sample code above, it flags the types from blob.cpp as undefined. What might be the cause of this? This may be a rudimentary Visual C++ issue, as I’m still new to this development platform. Thanks in advance for any help you can offer!

  10. I’m hitting the same problem as a few others in this thread, where, having followed the directions in the thread, I hit the following error upon compilation with Visual C++ 2010 Express:

    “LINK ; fatal error LNK1104: cannot open file ‘mfc100d.lib'”

    If anyone knows what might be causing this, please don’t hesitate to let me know! Thanks in advance for any assistance you can offer.

    • Problem solved. Some diligent Googling saved the day. As I was trying to run the blob code on Visual C++ Express 2010, the compiler was unable to locate the mfc lib file that I mentioned above because Microsoft’s Express software doesn’t allow mfc functionality. Fortunately I’m a university student, so I was able to acquire a free license to use Visual Studios 2010, but those of you without that good fortune may have to buy a more comprehensive software package if you hit the same problem.

      • Thanks for the feedback. I always took it for granted that I was using the full Visual Studio license. This kind of stuff is always useful for other visitors to the site. Regards,
        Andy

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>