AI Image tagging to simplify image management and search

It is popularly argued that vision is our most critical sense. We gather more information by seeing things than the other senses. And correspondingly, the human brain also retains a large portion of the information it gathers visually.

With the advancement in Artificial Intelligence, specifically in computer vision, software programs can now see and make sense of the world just like human vision. This advancement can be put to great use when we organize the digital images for our business.

And that is what we will look at in this blog, where we leverage AI for tagging images with information about their content, to help manage and search them better.

Why do we need image tagging?

Before we jump to leveraging AI image tagging, we must first understand why we need image tagging at all.

Over the past few years, as more and more users have started using the internet, businesses have increased their investment in establishing an online presence and brand, across multiple platforms and channels.

This has led to an explosion in the creation of digital content, especially images.

For example, e-commerce business will have thousands of product images, sales banners, images that go up on Facebook or Instagram, and more. To manage this extensive repository of images, we need better methods to organize them than just naming conventions or placing them in folders.

Tags provide us with the additional organizational and information layer necessary for managing our image repository.

All leading Digital Asset Management systems, including ImageKit, provide options to tag images. Using this information makes it easier for us to find the exact content amongst the thousands of images.

A common way of leveraging tags is to associate the information about the image's content, i.e., what you see in the image.

For example, you can tag the image below with tags like "Mobile Phone", "Plant", "Smartphone", etc.

These tags describe the image's actual content, which is far easier to remember, given human capabilities, than the file name or the folder it could be in.

The file might be named 123.jpg, but you would still be able to find an image with a "mobile phone" and a "plant" with the help of image tagging.

Manual tagging versus AI image tagging

There are two ways to add tags to the image.

You could manually tag the images, i.e., a person looks at an image, one at a time, and adds tags to it in your digital asset management system.

While this is great if you have a handful of images, this method is not scalable if you are dealing with an extensive image repository. It will take a lot of time and workforce to tag images manually.

And the second and more significant problem would be maintaining consistency and avoiding errors while tagging. Each individual could add tags to an image basis their vocabulary or interpretation of the image. The image below could be tagged as a "Car", a "Red car", or a "Luxury Car".

All are correct but may not be consistent with business requirements.

All tags are correct but are not consistent for a business

Therefore, a lot of effort would go into training a large team to maintain a consistent tagging vocabulary as a business.

There could also be manual spelling errors which would effectively render tagging useless as you would not be able to search for an image with that tag.

A much better alternative is provided by Artificial Intelligence, with the advancement in Computer Vision where computer software can see and interpret an image as humans do, identify objects in it, and label them automatically.

For example, Google Cloud Vision adds tags like "Wheel", "Tire", "Vehicle", "Car", and "Plant" to the same image as above, in milliseconds.

Example of AI image tags using Google Cloud Vision

Because software programs do this, AI image tagging is done in milliseconds and can be done for thousands of images simultaneously, making it exponentially faster than manual tagging.

Also, the same AI software would always generate the same tags for the same image, therefore taking away human inconsistency from image tagging. You save time on hiring and training a large workforce, and the ability to tag images with AI, in real-time and consistently, helps create better business workflows.

How you can start using AI image tagging for images

ImageKit is a complete image and video management and delivery solution. It comes with a rich digital asset management tool, called the Media library, to store and manage your images.

ImageKit's Media Library natively integrates with leading AI image tagging services like Google Computer Vision and AWS Rekognition. For any image that you upload to ImageKit, you can trigger either or both of these services to tag the image with minimal effort.

These tags are then associated with your images, and with ImageKit's advanced search can be used to search for images.

Let's look at how AI image tagging works in ImageKit and simplifies searching for images.

AI tagging of assets in ImageKit

Tagging images at the time of image upload

We would look at tagging images via APIs to integrate them in your workflow programmatically.

In ImageKit's file upload API, you can optionally pass a parameter to leverage either Google Cloud Vision or AWS Rekognition for tagging the image.

Here is a sample code to use both of these extensions at the time of image upload and get 5 tags from each.

curl -X POST "https://upload.imagekit.io/api/v1/files/upload" \
-u your_private_api_key: \
-F 'file=@/Users/username/Desktop/watch_picture.jpg;type=image/jpg' \
-F 'fileName=my_file_name.jpg'
-F 'extensions="[
  {
      \"name\": \"google-auto-tagging\",
      \"minConfidence\": 50,
      \"maxTags\": 5
  },
  {
      \"name\": \"aws-auto-tagging\",
      \"minConfidence\": 50,
      \"maxTags\": 5
  },
]"'

You can also add AI-generated tags with the Update image API as described in the documentation.

Once the image gets uploaded, in the API response, you get the uploaded image's URL, along with the tags generated by the selected AI service. The tags are provided as an array of objects as shown below.

{
    "fileId" : "598821f949c0a938d57563bd",
    "name": "my_file_name.jpg",
    "url": "https://ik.imagekit.io/your_imagekit_id/my_file_name.jpg",

	-----other fields------

    "AITags": [{
        "name": "Wristwatch",
        "confidence": 98.17,
        "source": "aws-auto-tagging"
    },
    {
        "name": "Person",
        "confidence": 97.72,
        "source": "aws-auto-tagging"
    },
    {
        "name": "Wrist",
        "confidence": 94.4,
        "source": "aws-auto-tagging"
    },
    {
        "name": "Hand",
        "confidence": 94.4,
        "source": "aws-auto-tagging"
    },
    {
        "name": "Digital Watch",
        "confidence": 72.91,
        "source": "aws-auto-tagging"
    },
    {
        "name": "Watch",
        "confidence": 98.29,
        "source": "google-auto-tagging"
    },
    {
        "name": "Hand",
        "confidence": 95.68,
        "source": "google-auto-tagging"
    },
    {
        "name": "Communication Device",
        "confidence": 86.49,
        "source": "google-auto-tagging"
    },
    {
        "name": "Finger",
        "confidence": 85.43,
        "source": "google-auto-tagging"
    }]
}

And that's it. These tags are now associated with your image. As explained later in this blog, you can also use them to search for images. A simple image upload API with an additional parameter lets you leverage AI image tagging for your images.

Ensuring we are getting only the "right" tags

When an AI algorithm looks at an image and tries to tag it, there is a score associated with how confident the software is about that tag. Yes, there will be tags that even Google or AWS's service might not be very confident about.

For example, in the wristwatch image above, tags such as "Finger", "Communication Device", and "Digital Watch" have low confidence scores as compared to "Wristwatch", "Watch", and "Hand" and rightly so.

We do not want to associate irrelevant tags with our images as that will lead to more clutter than organization.

ImageKit allows you to set the minimum confidence threshold when using the AI image tagging services using the minConfidence parameter. In the upload API example above, we used a threshold of 50. We can increase it to an integer value up to 100, with 100 indicating the highest possible confidence.

Only the tags with a confidence score above this threshold will be associated with the image you upload.

By setting the confidence threshold at 80, we can discard a lot of vague tags, retaining the more relevant ones. For example, tags with a confidence score below 80 are not relevant to the content in the image below.

Searching for images tagged using AI

Just tagging an image with its content is of no utility on its own. The ability to search for an image based on the tags associated with it makes tagging useful.

ImageKit provides a List and Search API that allows you to search for files based on different parameters, including its tags.

If we want to find all watches in our system, and all our images are tagged, we can search for the tag 'Wristwatch' via the API and get a list of all matching images.

curl -X GET 'https://api.imagekit.io/v1/files' \
-G --data-urlencode "searchQuery=\"tags\" IN [\"Wristwatch\"]" \
-u your_private_api_key:

The result would be a list of image objects, with each object containing all the information associated with that image. In our case, we had only one image with this tag, which gets returned in the response.

[
   {
      "type":"file",
      "name":"wrist_watch.jpg",
      "createdAt":"2021-12-11T01:17:12.544Z",
      "updatedAt":"2021-12-11T01:17:13.901Z",
      "fileId":"61b3fc181409db0f3da8c0c1",
      "AITags":[
         {
            "name":"Watch",
            "confidence":98.29,
            "source":"google-auto-tagging"
         },
         {
            "name":"Hand",
            "confidence":95.68,
            "source":"google-auto-tagging"
         },
         {
            "name":"Communication Device",
            "confidence":86.49,
            "source":"google-auto-tagging"
         },
         {
            "name":"Finger",
            "confidence":85.43,
            "source":"google-auto-tagging"
         },
         {
            "name":"Gesture",
            "confidence":85.26,
            "source":"google-auto-tagging"
         },
         {
            "name":"Wristwatch",
            "confidence":98.17,
            "source":"aws-auto-tagging"
         },
         {
            "name":"Person",
            "confidence":97.72,
            "source":"aws-auto-tagging"
         },
         {
            "name":"Wrist",
            "confidence":94.4,
            "source":"aws-auto-tagging"
         },
         {
            "name":"Hand",
            "confidence":94.4,
            "source":"aws-auto-tagging"
         },
         {
            "name":"Digital Watch",
            "confidence":72.91,
            "source":"aws-auto-tagging"
         }
      ],
      
      ---- other data ----
      "url":"https://ik.imagekit.io/ikmedia/ai_image_tagging/wrist_watch.jpg",
      
      ---- other data ----
      "height":4160,
      "width":6240,
      "size":1221626
   }
]

Combining AI image tagging with ImageKit's advanced search makes image search truly visual and will save your teams a ton of time finding the right content.

Perfecting the image tagging system - AI tagging with a manual review

While AI solutions will generate tags that are almost always right, not all tags may suit your business.

For example, Google Cloud Vision generates the tags, "Tire", "Wheel", "Vehicle", and "Car", for the following image. While both are correct visually, your business might just want to have "Car" as the tag associated with it. Therefore, a manual review of the tags generated might be helpful in such cases.

Selecting image tags that are relevant to your business

ImageKit allows you to remove a particular tag from a file. You can remove the tags "Tire" and "Wheel" from the image using the remove AI tag API as shown below.

curl -X POST "https://api.imagekit.io/v1/files/removeAITags" \
-H 'Content-Type: application/json' \
-u your_private_key: -d '
{
	"fileIds" : [
		"5e21880d5efe355febd4bccd" //file ID of the image in ImageKit
	],
	"AITags" : [
		"Tire", 
		"Wheel"
	]
}

Wrapping it up

Advancements in AI now allow us to use human vision-like capabilities in our business.

Leveraging this technology with DAM solutions like ImageKit, to tag our images with the image's actual content can simplify asset management and search to a great extent.

You can start using AI image tagging for image management and search for absolutely free with ImageKit and its integrated AI tagging extensions of Google Cloud Vision and AWS Rekognition.

推荐订阅源

ImageKit.io Blog