![\"Come](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/born_this_way.gif\")
<\/p>\n
I wrote my first blog post about how I got into programming (see here<\/a>) and how you can do that too, even when you’re not from the IT-field. In this post, we will play around with OpenCV in Android. Since I didn’t have a small use-case in mind for the vast feature-list of OpenCV I’ll keep that part simple.<\/p>\n For my first app idea, reading business cards could be a neat side-feature, and as I heard that OpenCV and Tesseract are a powerful team – process pictures with OpenCV and read texts with Tesseract OCR – I saw an excellent opportunity to put both into a thin app. OpenCV is an open-source computer vision library. It has a mighty feature-set, starting with simple image processing over object-, face- and feature recognition, 3d-calibration, and visualization up to machine learning and more. There is a big community with scientists, professionals, and hobbyists maintaining a comprehensive documentation – helpful for everyone, including beginners (see here<\/a>). I was more than happy realizing that it supports Android development, next to C++, C, Python, Scala, Java. Unfortunately, the inbuilt OCR is not known to be very mighty, therefore I decided to combine the strengths of OpenCV’s image processing with another library, called Tesseract (as a Marvel Fan I really like that name).<\/p>\n Tesseract is a free software, command line program for Linux, Windows, and next to the support of 100+ different languages, it was considered the most accurate open-source OCR available in 2006. In 2011 a brave man called Robert Theis released a fork version called “tess-two” of Tesseracts OCR Android tools, maybe not knowing that just 2 years later no one would work on Tesseract Android Tools anymore (the last commit was made 2013). Thanks to Robert Theis we can enjoy the opensource library up till today, updated, with new features and easy to use. So let’s get to work and get the setup of your Android Studio done:<\/p>\n To be able to use OpenCV in its pure form we first should setup Android NDK properly, and yes… we’re gonna write C++-Code in our App \ud83d\ude09 After creating a new Empty Activity Project use the Android Studio SDK Manager ( Next time you wake up from sweet dreams maybe the download is over and you can keep working. Now you need to add the location of your Android NDK. Write it into There you go. Your first step to glory is over. The next stuff is going to be a bit more tricky. OpenCV – here we come!<\/p>\n To setup OpenCV, the first step is to download the latest release of the library from their page: OpenCV Releases<\/a> (at this point it’s 3.4.2). Unpack the zip, preferably into the Android SDK folder (so you have it all in one place) and import that stuff, as a module, into your project: Hit “Next” and “OK” as long as they ask you stuff, keep going with the default checkmarks until you finally finish. Now, add the module as a dependency to your app. At Next step is to correct the Now you create a directory in your project src\/main\/jni. In there you create a file called Write the following code into the Write your Android NDK path into Now you can tell your Fill the parameter the following After successfully setting javah up, write a little test function into your Use the javah-tool via To use the camera we’re going to implement some code. Initially we implement our MainActivity and its XML-structure. This is going to be a small button on an empty screen, nothing more. But there is more to come! I prepared a gist for you, where you can simply copy\/paste the code into the corresponding files: MainActivity for initial setup<\/a> (don’t forget to change the package name)<\/p>\n Our next step is to create a new empty Activity for working with the camera, let’s call it As you can see, we really should take care of state changes, due to the fact that the camera is pretty demanding to run constantly. New to you will be the Well done! You should be able to run the app now. And if you press the button… wait! What is this? Ahhh!! We forgot the permissions, can it be it’s the permissions?<\/p>\n So for getting the permissions, we will check and if necessary ask for them in the The last step to get the permissions is to touch the Okay, let’s see if this was the mistake and run the app…Asking for permissions seems to be working but… Oh come on! Nooooo!! Press “No”! We don’t need that ominous “OpenCV Manager”! We already have the whole library inside our app.<\/p>\n Ok ok…calm down. We can easily correct the rude OpenCV behavior:<\/p>\n So one of the most useless features of OpenCV is, that it checks if the device it’s running on has the OpenCV library installed, and if not, it won’t work until the OpenCV Manager App is installed. Since we put the whole library into the actual app, we don’t need that anymore, and yet, due to the fact that the library isn’t loaded completely when it checks it causes this message. With the following gist, we override the Now run the app again. If you didn’t mix up some code it should finally look like this:<\/p>\n Good job! But no time to celebrate: Somehow the camera orientation and size makes you feel a bit dizzy, right? We are getting our hands dirty and change something right inside the OpenCV library to fix this one. Find the class Look for the method Yeah! How wonderfully straight it looks like this, right? It somehow feels like the first time we see stuff through our camera. If your camera view seems a bit too small or big, you can simply change the mScale value accordingly. If you made it till here, congratulations! Figuring out the whole procedure till here cost me around 2 days of my life! I’m glad to make sacrifices for you, though \ud83d\ude09 The next part is going to be much easier!<\/p>\n Let’s start putting some functionality into our app and add some more buttons to our Don’t mind the errors yet, we will get to those now. Create a Java class with the name Now delete the previously generated header file, from as you set up the Navigate to your Then, inside of the jni-folder, create a And last but not least, copy the name of this Run your app and…tadaaa! Now you can play around with four of the most common functions of OpenCV. Well done!<\/p>\n You just learned how to use C++-code AND OpenCV functionality inside your Android app. If this is all you wanted to learn, give yourself a break and lean back. Give your beloved ones a hug and tell them, you did it! Also, give yourself a break when this took you too long and your computer is smoking from all the building processes. In times like these, during the hot summer, I prefer to put my laptop into the fridge to give it a little chill \ud83d\ude42 Now that we had a wonderful bath in a jacuzzi full of self-esteem, we can go back to work. There is one little part missing: The OCR! Don’t worry, I will make it quick. Tess-two is now available and ready to use. Add the HUD components to Create a new Java class with the name Let’s use it in the Tesseract, and therefore tess-two, needs traineddata to know how and what to recognize. Since we are using tess-two, which is a fork of an older Tesseract version, we cannot just use any traineddata that is available in the tesseract git repository. Download the following traineddata and you will be fine: Download eng.traineddata<\/a>. Create an asset-folder via Next, you need to make sure that this file is also put onto the device. Therefore we need to add a little method to our Alright! The last thing to do is to get the permissions for read and write access on the external storage since we are doing that with our previously added method. Go and add the following checks and constants to the Okay! Breath in. Breath out. You did it! Build (can take a while), run and play \ud83d\ude42 Hold your phone in front of the text below and press Find my original Git repo of the app here<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
\nIn this blog post, I will show you how to write that app all by yourself, step by step. You will learn to use C++-code, OpenCV and Tesseract OCR on Android.<\/p>\nWhat is OpenCV?<\/h3>\n
![\"Loki\"](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/marvel_fan.gif\")
<\/p>\nWhat is Tesseract OCR?<\/h3>\n
OpenCV Part<\/h2>\n
Setup Android NDK<\/h3>\n
![\"AAAHHH\"](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/ahhh_funny.gif\")
<\/p>\nTools -> SDK Manager -> SDK Tools<\/code>) for installing the necessary Tools: NDK, CMake, and LLDB. Mind the location of your SDK though – you’ll need that for the next step. After applying your changes a long download of around 700MB will start – don’t worry, that is as intended.
\n![\"sdktools_setup\"](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/sdktools_setup.png\")
<\/p>\nFiles -> Project Structure -> SDK Location -> Android NDK<\/code> location. Yes, you just saw the needed path: it is the SDK location + \/ndk-bundle<\/code>.<\/p>\n![\"Android](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/android_ndk_loc.png\")
<\/p>\nSetup OpenCV<\/h3>\n
File -> New -> Import Module<\/code>. Navigate in the next dialogue that pops up to the java-folder of the just unpacked module, i.e. mine is \/home\/soeren\/Downloads\/OpenCV-android-sdk\/sdk\/java<\/code> (on Linux). If you’re correct the name of the library will show up and it looks like the following (ignore “Project already….” in that pic):<\/p>\n![\"import](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/import_module_window.png\")
<\/p>\nFile -> Project Structure -> app -> Dependencies -> + -> Module dependency<\/code> you will find OpenCV shown in a pop-up.<\/p>\n![\"add](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/add_module_dependency.png\")
<\/p>\nbuild.gradle<\/code>-file of the OpenCV Library (you will find it right under your app build.gradle<\/code>-file). Align the fields compileSdkVersion<\/code>, minSdkVersion<\/code>, and targetSdkVersion<\/code>, so they are matching with the build.gradle<\/code>-file (app). Mine looks like this:
\nbuild.gradle (openCVLib…) for initial setup<\/a><\/p>\nAndroid.mk<\/code> and another called Application.mk<\/code> – to tell the NDK tool to build native code.<\/p>\n![\"where](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/where_to_put_mks.png\")
<\/p>\nAndroid.mk<\/code>-file:
\nAndroid.mk for NDK and OpenCV setup<\/a><\/p>\nNDK_MODULE_PATH<\/code> and replace the OPENCV_ROOT<\/code> according to yours. Later we will add the LOCAL_SRC_FILES<\/code> name, but for now, we can leave it empty. Then modify the Application.mk<\/code>-file:
\nApplication.mk for NDK setup<\/a><\/p>\nbuild.gradle<\/code>-file (app) to compile C++-code to .so<\/code>-files into the jniLibs folder. Do that by writing this…
\nbuild.gradle (app) File for OpenCV setup<\/a>
\n…into your build.gradle (app)<\/code> file, by overwriting the default buildTypes-part. Remember to change the ndkDir value to your own. Don’t hesitate to process the gradle-sync, but if you try to build your app now you will get a wonderful error. Something like Process 'command 'PATHTONDK\/ndk-build'' finished with non-zero exit value 2\"<\/code>. This cryptic error messages did cost me quite some patience, but don’t worry. This is still as expected because we didn’t give in any LOCAL_SRC_FILES<\/code>. That’s what we’re going to do now. For the sake of convenience, we’re going to use javah as an external tool. For doing so, navigate to File -> Settings -> Tools -> External Tools -> +<\/code>. Add the javah tool like this:<\/p>\n![\"javah](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/javah_setup.png\")
<\/p>\n
\n* “Program”: the path to your javah installation (with Linux you can find out the path typing which javah<\/code> into a terminal)
\n* “Arguments”: -v -jni -d $ModuleFileDir$\/src\/main\/jni $FileClass$<\/code>
\n* “Working directory”: $SourcepathEntry$<\/code><\/p>\nMainActivity<\/code>:<\/p>\npublic static native int test();\r\n<\/code><\/pre>\nright-click on test()-> Android Tools -> javah<\/code> and you generate a corresponding header file and can find it in your jni-folder. Copy the name of that file into your Android.mk<\/code>-file to LOCAL_SRC_FILES<\/code>, mine is called com_example_android_MYAPPLICATION_MainActivity.h<\/code>, yours will be something close to that, I guess.
\nNow, run and pray that you didn’t make any path- or copy-paste-mistake. If something is not working and you get a cryptic error-message – check all the paths again, and when that doesn’t help, start over again – been there… done that… can happen to everyone! Just don’t give up. \ud83d\ude42
\nIf it builds successfully and a “Hello World!” is smiling from your Device-Screen up to you: YEAH!!! You’ve done great so far. The actual fun starts now!<\/p>\nSetup the Camera<\/h3>\n
CameraActivity<\/code>. Here we will call the camera and have fun with OpenCV and tess-two later, but for now, we will only call the camera. Copy the code from the following gist into the corresponding files: CameraActivity initial setup<\/a>.<\/p>\nonCameraViewStarted<\/code>, onCameraViewStopped<\/code> and onCameraFrame<\/code>-methods. Here you can directly interact with the camera input.
\nIn our MainActivity<\/code> we will call the CameraActivity<\/code>, by replacing the \/\/ do stuff here<\/code> with the following code:<\/p>\nIntent intent = new Intent(MainActivity.this, CameraActivity.class);\r\nstartActivity(intent);\r\n<\/code><\/pre>\n![\"OpenCV](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/openCV_error_msg.jpg\")
<\/p>\nGet the permissions for OpenCV<\/h3>\n
MainActivity<\/code>, when the user presses the button. Modify the MainActivity<\/code> with all the uncommented code at their proper position (as shown with the comments): Permissions in MainActivity<\/a><\/p>\nAndroidManifest.xml<\/code>: Manifest for camera permissions<\/a><\/p>\n![\"Package](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/package_not_found.jpg\")
<\/p>\n![\"oh](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/ohcomeon.gif\")
<\/p>\nIgnore OpenCVManager<\/h3>\n
onPackageInstall<\/code>-method of the BaseLoaderCallback<\/code> class to prevent the app to trigger the installation. Add the following uncommented code into your CameraActivity<\/code> right into the BaseLoaderCallback<\/code> class:
\nBaseLoaderCallback-class override the onPackageInstall-method<\/a><\/p>\n![\"First](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/firstcam_shot-e1533581737932.jpg\")
<\/p>\nCameraBridgeViewBase<\/code> here:<\/p>\n![\"CameraBridgeViewBase](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/camerabridgeviewbase_location.png\")
<\/p>\ndeliverAndDrawFrame<\/code> and replace it with the following code: Adjusting rotation and scale of the camera<\/a><\/p>\nUse the OpenCV library for image processing<\/h3>\n
activity_camera.xml<\/code> and corresponding listeners to our CameraActivity<\/code>:
\nAdding function buttons, listeners and call OpenCvMaker methods<\/a><\/p>\nOpenCvMaker.java<\/code> and add the following code (and again, remember to change the package name):
\nCreate an OpenCV manager class<\/a><\/p>\nnative test()<\/code> method with the javah tool.<\/p>\n![\"Generated](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/generated_header_file.png\")
<\/p>\nOpenCvMaker<\/code> class, use the javah tool on one of the native methods (i.e. makeGray(long......)<\/code>) with right click -> Android Tools -> javah<\/code>. A new header file will get generated. Go to this file and make the following changes:
\nAdd methods to the header<\/a><\/p>\n.cpp<\/code>-file (without header) and call it the same as your header and add the following code to it:
\nImplementation of OpenCV methods in C++<\/a><\/p>\n.cpp<\/code>-file (including the “.cpp<\/code>“!!) and set it as your LOCAL_SRC_FILES<\/code> inside your Android.mk<\/code>-file.<\/p>\n![\"celebrations\"](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/celebrate.gif\")
<\/p>\n
\n![\"my](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/computer_in_fridge.jpg\")
<\/p>\nTess-Two Part<\/h2>\n
\nFirst, add the following dependency to your build.gradle (app)<\/code>-file and trigger the gradle-sync:<\/p>\nimplementation 'com.rmtheis:tess-two:9.0.0'\r\n<\/code><\/pre>\nactivity_camera.xml<\/code> and some functionality to CameraActivity.java<\/code> with the following modifications:
\nRead-button and textview for CameraActivity<\/a><\/p>\nMyTessOCR.java<\/code> and put the following code into it (don’t forget to change the package!):
\nImplementing the tess-two class<\/a><\/p>\nCameraActivity<\/code>, and make some changes in the CameraActivity<\/code>. Since computation of the OCR could be process-demanding, we will call it in an extra thread. Also, you will be asked to import the android.graphics.Matrix<\/code>, continue and confirm. This is only for rotating the input signal of the camera according to the preview, that we’ve rotated inside the OpenCV library already. Do the following: Call MyTessOCR class from CameraActivity<\/a><\/p>\nright-click on Java -> New -> Folder -> Asset Folder<\/code>, then create a directory called tessdata<\/code> and finally copy the downloaded eng.traineddata<\/code>-file into that directory. When you’re done it should like this:<\/p>\n![\"traineddate](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/traineddata_location.png\")
<\/p>\nCameraActivity<\/code> and call it in the onCreate method: Add preprateTessData-method to CameraActivity<\/a><\/p>\nMainActivity<\/code> and additional permissions to the AndroidManifest.xml<\/code>:
\nAsk for external storage read and write permissions<\/a><\/p>\nREAD IT!<\/code> in your app.<\/p>\n![\"Congratulations\"](\"https:\/\/craftcoders.app\/wp-content\/uploads\/2018\/08\/congratulations.png\")
<\/p>\n