SmartCrop smart crop correction

Preface

From the previous two articles, we learned about two types of image tilt: plane tilt and Z-axis tilt, and combined with the previously learned API to complete the ideal image correction work. However, the image tilt correction based on the ideal picture does not play a big role in the actual development process. The author's abilities are limited, so I can't help you realize the Android application with strong corrective ability and easy to use at present. However, we can stand on the shoulders of giants to achieve our goals.

SmartCrop

SmartCrop is a simple and easy-to-use smart picture cropping library suitable for cropping photos of ID cards, business cards, documents, etc. It was discovered by the author accidentally while traveling on Github, and it just made up for the shortcomings of the previous two articles that were not practical.

Source address:

https://github.com/pqpo/SmartCropper

Features

  • Use smart algorithms (based on OpenCV) to identify the borders in the picture
  • Support dragging the anchor point, manually adjust the selection, and the magnifying glass effect enhances the positioning experience
  • Use perspective transformation to crop and correct the selection to restore the front image
  • Support rich UI settings, such as auxiliary lines, masks, anchors, magnifiers, etc.

principle

SmartCrop uses a machine learning-based HED network to optimize the edge detection effect. The specific algorithm process reference: A practice of running a convolutional neural network on the mobile phone-based on TensorFlow and OpenCV to achieve the document detection function . In the article, the author first introduces the traditional technical solutions (Canny and findContours), then analyzes the difficulty and limitations of the traditional technical solutions, and then through repeated tuning and testing of the traditional solutions, the author concludes that the traditional algorithm methods have reached the limit, and then try Machine learning/neural network. Since the author is currently not familiar with machine learning and neural networks, and cannot help you solve the algorithm process, capable friends can consult the original text by themselves.

integrated

Add dependency

  • Project root directory build.gradle
allprojects {
    repositories {
        google()
        jcenter()
        maven(url = "https://jitpack.io") // 添加 jitpack maven 仓
    }
}
  • Application directory build.gradle
implementation("com.github.pqpo:SmartCropper:v2.1.3")

Configuration

  • Application directory build.gradle
android {
    ……
    // 不压缩模型
    aaptOptions {
        noCompress("tflite", "lite")
    }
}
  • Application directory proguard-rules.pro
-keep class me.pqpo.smartcropperlib.**{*;}

step

  • Application.onCreate In-method initialization
override fun onCreate() {
    super.onCreate()
    ……
    SmartCropper.buildImageDetector(this)
}
  • Adding CropImageViewto the layout
<me.pqpo.smartcropperlib.view.CropImageView
    android:id="@+id/iv_crop"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />
Note: CropImageView inherits from ImageView, but ScaleType must be a centered type. If you manually set it to fit_end, fit_start, matrix, an error will be reported.
  • Set up a picture to be processed
mBinding.ivCrop.setImageToCrop(selectedBitmap)
This method uses native code to intelligently identify the border, and draws pictures and selections. Implemented in the native layer, greatly improving the operating efficiency, the running time is proportional to the size of the picture, in the case of large pictures, you can consider executing in a sub-thread, or compressing the incoming picture.
  • Crop the picture
mBinding.ivShow.setImageBitmap(cropBitmap)
Crop out the pictures in the selection area according to the selection area, and use perspective transformation to correct it into a frontal picture.
Note: The main logic of the modified method is also located in the native layer, and the running time is proportional to the size of the picture. In the case of large pictures, you can consider executing it in a sub-thread or compress the incoming picture.

Code

  • layout
<?xml version="1.0" encoding="utf-8"?>
<layout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto">

    <data>

    </data>

    <androidx.constraintlayout.widget.ConstraintLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent">

        <FrameLayout
            android:id="@+id/container"
            android:layout_width="match_parent"
            android:layout_height="0dp"
            app:layout_constraintBottom_toTopOf="@id/bt_select"
            app:layout_constraintTop_toTopOf="parent">

            <me.pqpo.smartcropperlib.view.CropImageView
                android:id="@+id/iv_crop"
                android:layout_width="match_parent"
                android:layout_height="match_parent" />

            <ImageView
                android:id="@+id/iv_show"
                android:layout_width="match_parent"
                android:layout_height="match_parent"
                android:visibility="gone" />
        </FrameLayout>

        <Button
            android:id="@+id/bt_select"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:onClick="pickImage"
            android:text="选择图片"
            app:layout_constraintBottom_toBottomOf="parent"
            app:layout_constraintLeft_toLeftOf="parent"
            app:layout_constraintRight_toLeftOf="@id/bt_crop"
            app:layout_constraintTop_toBottomOf="@id/container" />

        <Button
            android:id="@+id/bt_crop"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:onClick="cropImage"
            android:text="裁剪图片"
            android:visibility="gone"
            app:layout_constraintBottom_toBottomOf="parent"
            app:layout_constraintLeft_toRightOf="@id/bt_select"
            app:layout_constraintRight_toRightOf="parent"
            app:layout_constraintTop_toBottomOf="@id/container" />

    </androidx.constraintlayout.widget.ConstraintLayout>
</layout>
  • Business logic
/**
 * 智能裁剪
 * author: yidong
 * 2021-06-04
 */
class SmartCropActivity : AppCompatActivity() {


    private val mBinding: ActivitySmartCropBinding by lazy {
        ActivitySmartCropBinding.inflate(layoutInflater)
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(mBinding.root)
    }

    fun pickImage(view: View) {
        hideCropButton()
        hideCroppedImage()
        EasyPhotos.createAlbum(
            this, true, false,
            GlideEngine.instance
        )
            .setFileProviderAuthority(BuildConfig.APPLICATION_ID)
            .setCount(1)
            .start(object : SelectCallback() {
                override fun onResult(photos: ArrayList<Photo>, isOriginal: Boolean) {
                    Log.d(App.TAG, photos.toString())
                    if (photos.isNotEmpty()) {
                        val path = photos.first().path
                        val options = BitmapFactory.Options()
                        options.inJustDecodeBounds = true
                        BitmapFactory.decodeFile(path, options)
                        options.inJustDecodeBounds = false
                        options.inSampleSize = calculateSampleSize(options)
                        val selectedBitmap = BitmapFactory.decodeFile(path, options)
                        if (selectedBitmap != null) {
                            mBinding.ivCrop.setImageToCrop(selectedBitmap)
                        }
                        showCropButton()
                    }
                }

                override fun onCancel() {}
            })
    }

    fun cropImage(view: View) {
        showCroppedImage()
        hideCropButton()
        if (mBinding.ivCrop.bitmap != null) {
            val cropBitmap = mBinding.ivCrop.crop()
            mBinding.ivShow.setImageBitmap(cropBitmap)
        }
    }

    private fun calculateSampleSize(options: BitmapFactory.Options): Int {
        val outHeight = options.outHeight
        val outWidth = options.outWidth
        var sampleSize = 1
        val destHeight = 1000
        val destWidth = 1000
        if (outHeight > destHeight || outWidth > destHeight) {
            sampleSize = if (outHeight > outWidth) {
                outHeight / destHeight
            } else {
                outWidth / destWidth
            }
        }
        if (sampleSize < 1) {
            sampleSize = 1
        }
        return sampleSize
    }

    fun showCropButton() {
        mBinding.btCrop.visibility = View.VISIBLE
    }

    private fun hideCropButton() {
        mBinding.btCrop.visibility = View.GONE
    }

    fun showCroppedImage() {
        mBinding.ivShow.visibility = View.VISIBLE
        mBinding.ivCrop.visibility = View.GONE
    }

    private fun hideCroppedImage() {
        mBinding.ivShow.visibility = View.GONE
        mBinding.ivCrop.visibility = View.VISIBLE
    }
}

effect

effect

Source code

https://github.com/onlyloveyd/LearningAndroidOpenCV