The Task: What’s the name of that flower?
We want to build an app that recognizes flowers using the cell phone camera. The workflows is that:
- We take a picture of a flower using the cell phone camera
- We match the picture against a large database of images of flowers and their names
- We output the name of the best matching flowers
Building a database of flower images
We use the bing search engine api on Azure to gather images. First, we create a list of flower names. Each flower can have multiple names, for example one in Latin and one in English. We can also construct multiple queries for disambiguation. The following are reasonable queries:
We use bing image search to retrieve the urls of images of flowers on the web. At a later stage we use a script to download those images.
Issues: Sometimes the bing search for flower name retrieves cartoon characters (because those are named after a flower), drawings of flowers rather than pictures. Some files are too large and it makes little sense to download and process them.
- bing image search
- download images
Image search is usually based on explicit features extracted from the images. Those features are in the following categories:
Color based features, usually a normalized color histogram
Texture based features
Practical Stuff: OpenCV with Python
It seems that the easiest way to build an image search is to use OpenCV from Python (import cv2 library)
One needs to install this library from source. The latest version is 3.0.
This is a bit tricky.
First one needs to decide if the stable or the development version will be used.
- Install the prerequisites:
[compiler] sudo apt-get install build-essential [required] sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev [optional] sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
Decide if you want only python2 support (old Python) or Python3 support. If python3 is required install python3 and python3-dev (python3-numpy?). I built without Python3
Prepare the arguments to cmake
cmake CMAKE_BUILD_TYPE=RELEASE \ -D BUILD_PYTHON_SUPPORT=ON \ -D BUILD_EXAMPLES=ON \ -D INSTALL_C_EXAMPLES=ON \ -D INSTALL_PYTHON_EXAMPLES=ON \ -D BUILD_NEW_PYTHON_SUPPORT=OFF \ -D CMAKE_INSTALL_PREFIX=
/opencvbin \ -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules \ ../opencv/ </pre> Important: also consult this page install-opencv-3-0-and-python-2-7-on-ubuntu. It suggests more libraries to install (depending on the purpose of usage), a fortan library for optimizing the routines and a virtual python environment.