在H5中实现OCR拍照识别身份证功能

业务背景

由于当前项目中需要实现身份证拍照识别的功能，如果是小程序可以使用微信提供的 ocr-navigator (opens new window) 插件实现,但是在企业微信的H5中没有提供该插件，所以需要手动实现该功能。

需求分析及资料查阅

众所周知，前端H5中浏览器打开相机打开的是原生相机，无法在相机的界面上覆盖自定义的元素，比如实现类似下面的UI界面,无法使用相机拍照功能来直接实现，所以只能另辟蹊径。

• 通过查阅资料发现，可以通过MediaDevices.getUserMedia() (opens new window)来实现媒体流的输出，这时可以在页面中添加video元素，然后把stream流的值赋值给video的srcObject属性，就可以把video输出到页面上，这样就可以在video元素上面添加自定义元素，实现UI效果。
• 还需要解决的问题是：如何点击下面的拍照按钮时把获取画面转换成图片，并调用Api实现图片识别功能。 此时需要使用canvas来实现。通过canvas将video视频的当前帧绘制到画布上，然后将其转换成图片，然后调用接口来实现身份证识别。

snapPhoto() {
  const canvas = document.querySelector("#mycanvas");
  canvas.width = this.video.videoWidth;
  canvas.height = this.video.videoHeight;
  canvas.getContext("2d").drawImage(this.video, 0, 0);
  const imageBase64 = canvas.toDataURL("image/png", 0.6);
  return imageBase64
}

需求实现

话不多说，直接上代码(注意：该页面代码 vue-cli3 + vue2 + vant + 企业微信环境)

<template>
  <div class="ocr-id-card">
    <div id="cover" class="cover">
      <div class="id-card-container"></div>
      <video ref="videoRef" class="media-video" autoplay playsinline></video>
    </div>
    <div class="footer-tip font-24 radius-32 color-fff flex-center">请将证件放于框内拍摄</div>
    <div class="footer-btn">
      <div class="album" @click="chooseLocalImage">
        <img src="@/assets/parttime-operator/album.png" alt="" class="album-img width-68 height-68" />
      </div>
      <div id="snap" class="record-btn" @click="snapPhoto"></div>
    </div>

    <canvas id="mycanvas" class="card-canvas"></canvas>
  </div>
</template>
<script>
import { uploadFileApi, idCardOcrApi } from "@/apis/common";
import { base64URLToFile } from "@/utils/base64-to-img";
export default {
  data() {
    return {
      image_url: "", // 身份证url
      imageBase64: "", // 身份证照片 base64
      cardSide: "FRONT", // 身份证正反面 FRONT：身份证有照片的一面（人像面）BACK：身份证有国徽的一面（国徽面
      video: {},
      videoTrack: {}
    };
  },
  mounted() {
    const { cardSide } = this.$route.query;
    this.cardSide = cardSide;
    this.watchPageVisible();
  },
  beforeRouteLeave(to, from, next) {
    if (this.videoTrack) {
      this.videoTrack.stop();
    }
    next();
  },
  methods: {
    // 调用摄像头
    openCamera() {
      // constraints: 指定请求的媒体类型和相对应的参数
      const constraints = {
        audio: false,
        video: {
          width: window.innerHeight * window.devicePixelRatio,
          height: window.innerWidth * window.devicePixelRatio,
          frameRate: { ideal: 60 }, // 视频流帧率
          facingMode: "environment" // 后置摄像头
        }
      };
      // 兼容部分浏览器
      if (!navigator.mediaDevices) navigator.mediaDevices = {};
      // 一些浏览器部分支持 mediaDevices,不能直接给对象设置 getUserMedia
      // 因为这样可能会覆盖已有的属性,只会在没有getUserMedia属性的时候添加它。
      if (navigator.mediaDevices.getUserMedia === undefined) {
        navigator.mediaDevices.getUserMedia = function(constraints) {
          // 首先，如果有getUserMedia的话，就获得它
          const getUserMedia =
            navigator.getUserMedia ||
            navigator.webkitGetUserMedia ||
            navigator.mozGetUserMedia ||
            navigator.msGetUserMedia ||
            navigator.oGetUserMedia;
          if (!getUserMedia) {
            return Promise.reject(new Error("getUserMedia is not implemented in this browser"));
          }
          // 否则，为老的navigator.getUserMedia方法包裹一个Promise
          return new Promise(function(resolve, reject) {
            getUserMedia.call(navigator, constraints, resolve, reject);
          });
        };
      }
      // 获取视频流
      navigator.mediaDevices
        .getUserMedia(constraints)
        .then(stream => {
          this.videoTrack = stream.getVideoTracks()[0];
          this.video = document.querySelector(".media-video");
          if (this.video) {
            this.video.srcObject = stream;
            this.video.onloadedmetadata = () => {
              this.video.play();
            };
          }
        })
        .catch(function(err) {
          // 用户拒绝打开摄像头的处理逻辑
          if (error instanceof DOMException && error.name === "NotAllowedError") {
            this.$toast("请允许打开摄像头完成拍照");
            this.videoTrack = null;
          } else {
            console.error(error);
          }
        });
    },
    // 监控页面visibilitychange
    watchPageVisible() {
      document.addEventListener("visibilitychange", () => {
        if (!document.hidden) {
          this.openCamera();
        } else {
          if (this.video && this.video.srcObject) {
            this.video.srcObject.getTracks().forEach(track => track.stop());
          }
        }
      });
    },
    // 获取视频的一帧作为图片转换为base64，调用接口识别身份证信息
    snapPhoto() {
      const canvas = document.querySelector("#mycanvas");
      canvas.width = this.video.videoWidth * 0.9;
      canvas.height = this.video.videoHeight * 0.9;
      canvas.getContext("2d").drawImage(this.video, 0, 0);
      const imageBase64 = canvas.toDataURL("image/png", 0.6);
      this.idCardRecognition(imageBase64);
    },
    // 身份证照片识别
    async idCardRecognition(imageBase64) {
      try {
        this.$toast.loading({
          duration: 0, // 持续展示
          message: "识别中...",
          forbidClick: true,
          loadingType: "spinner"
        });
        const params = { cardSide: this.cardSide, imageBase64 };
        const result = await idCardOcrApi(params);
        if (Object.keys(result).length) {
          const {
            Name,
            IdNum,
            ValidDate,
            AdvancedInfo: { IdCard }
          } = result;
          if (IdCard) {
            const imageBase64 = "data:image/png;base64," + IdCard;
            const file = await base64URLToFile(imageBase64);
            this.image_url = await this.uploadFile(file);
          }
          const id_card_end_time =
            ValidDate && ValidDate.indexOf("长期") === -1 ? ValidDate.split("-")[1].replace(/\./g, "/") : "";
          const id_card_info = {
            id_card_name: Name ? Name : "",
            id_card_num: IdNum ? IdNum : "",
            long_term: ValidDate ? (ValidDate.indexOf("长期") > -1 ? 1 : 2) : 0,
            id_card_end_time
          };
          if (this.cardSide === "FRONT") {
            id_card_info.id_card_front = this.image_url;
          } else {
            id_card_info.id_card_back = this.image_url;
          }
          this.$store.commit("COMMON/setIdCardInfo", id_card_info);
        } else {
          const file = await base64URLToFile(imageBase64);
          this.image_url = await this.uploadFile(file);
          const id_card_info = {};
          if (this.cardSide === "FRONT") {
            id_card_info.id_card_front = this.image_url;
          } else {
            id_card_info.id_card_back = this.image_url;
          }
          this.$store.commit("COMMON/setIdCardInfo", id_card_info);
        }
        this.$toast.clear();
        this.$toast({
          message: "识别成功",
          duration: 800,
          onClose: () => {
            this.$router.go(-1);
          }
        });
      } catch (err) {
        console.log(err);
      }
    },
    // 从相册选择图片
    chooseLocalImage() {
      // eslint-disable-next-line no-undef
      wx.chooseImage({
        count: 1,
        sizeType: ["compressed"],
        sourceType: ["album"],
        success: async res => {
          const id = res.localIds[0];
          // eslint-disable-next-line no-undef
          wx.getLocalImgData({
            localId: id,
            success: async res => {
              await this.idCardRecognition(res.localData);
              this.$toast.clear();
            },
            fail: err => {
              console.error("getLocalImgData err", err);
            }
          });
        }
      });
    },
    // 上传文件
    uploadFile(file) {
      return new Promise(async (resolve, reject) => {
        try {
          this.$toast.loading({
            message: "上传并识别中",
            forbidClick: true,
            loadingType: "spinner"
          });
          const params = new FormData();
          params.append("file", file);
          params.append("type", 1);
          params.append("file_name", file.name);
          const { url } = await uploadFileApi(params);
          resolve(url);
        } catch (err) {
          reject(err);
        }
      });
    }
  }
};
</script>
<style lang="less" scoped>
.ocr-id-card {
  width: 100vw;
  z-index: 2000;
  background: #fff;
  overflow: hidden;
  -webkit-overflow-scrolling: touch;

  .cover {
    width: 100vw;
    height: 100vh;
    position: fixed;
    top: 0;
    left: 0;
    z-index: 2001;

    .id-card-container {
      width: 708px;
      height: 460px;
      background: url("~@/assets/ocr-border.png") 0 0 no-repeat;
      background-size: 708px 460px;
      position: fixed;
      top: 322px;
      left: 50%;
      transform: translateX(-50%);
      z-index: 2004;
    }
  }
  .media-video {
    width: 100vw;
    height: 100%;
    position: absolute;
    top: 0px;
    left: 0;
    object-fit: cover;
  }
  .footer-tip {
    width: 312px;
    height: 64px;
    background: rgba(0, 0, 0, 0.5);
    position: fixed;
    bottom: 392px;
    left: 50%;
    transform: translateX(-50%);
    z-index: 2003;
  }
  .footer-btn {
    width: 100vw;
    height: 300px;
    background: #fff;
    position: fixed;
    bottom: 0;
    left: 0;
    z-index: 2005;
    .record-btn {
      width: 108px;
      height: 108px;
      background: url("~@/assets/take-photo.png") 0 0 no-repeat;
      background-size: 108px 108px;
      position: absolute;
      top: 76px;
      left: 50%;
      transform: translateX(-50%);
      z-index: 2006;
    }
    .album {
      width: 80px;
      height: 80px;
      position: absolute;
      top: 90px;
      left: 120px;
      z-index: 2006;
    }
  }
  .card-canvas {
    position: fixed;
    left: -9999px;
    top: -9999px;
    z-index: 0;
    backface-visibility: hidden;
    transform: translateZ(0);
  }
}
</style>

功能优化及兼容性bug修复

兼容性问题及注意点

1. 本地调试打开相机需要使用https协议下才能正常调用获取媒体流的api
2. ios环境下初次打开相机会展示直播界面，安卓系统正常
3. 媒体流帧率问题，视频分辨率问题，顶部空白问题。
4. ios有滚动条问题，安卓系统正常
5. 页面退出时关闭媒体流输入，关闭相机，进入时打开媒体流输入。

解决方案

• 本地开发时开启htpps

devServer: {
    https: true,
    xxx...
  }

• 页面中的元素使用fixed定位，并设置z-index高一些
• 设置视频流帧率和视频流的分辨率大小，使用视口宽度/高度 * 设备像素比

const constraints = {
    audio: false,
    video: {
      width: window.innerHeight * window.devicePixelRatio,
      height: window.innerWidth * window.devicePixelRatio,
      frameRate: { ideal: 60 }, // 视频流帧率
      facingMode: "environment" // 后置摄像头
    }
 };

• ios有滚动条问题

解决方案：设置媒体流视频的宽高后，设置视频宽高css,父盒子设置高度100vh,视频高度100%,设置object-fit: cover;

.media-video {
   width: 100vw;
   height: 100%;
   position: absolute;
   top: 0px;
   left: 0;
   object-fit: cover;
 }

• 调用ocr图片识别可以调用后端接口或者第三方的API来实现，例如腾讯云OCR (opens new window) 最后实现效果

参考文章链接

1. MediaDevices.getUserMedia: developer.mozilla.org/zh-CN/docs/… (opens new window)
2. H5实现自定义身份证拍照 juejin.cn/post/695503… (opens new window)