DetectObjects procedure

Official Content
This documentation is valid for:

Detects objects that appear in an image.

Parameters

Configuration

The following table resumes the configuration properties (access credentials) you must set in order to use this AI task.

PropertyKey
ProviderType Id Key SecretKey
Alibaba - - -
Amazon - Rekognition Rekognition
Baidu - - -
Google - Cloud Vision API -
IBM - - -
Microsoft - Computer Vision -
MLKit ML Kit API ML Kit API
SAP - - -
Tencent 物体识别 物体识别 -

Sample

Taking the following image input, the table below shows how objects are identified for each provider (as a JSON structure) and the time it takes for processing it.

GeneXusAI - Image module - Sample

Provider Output Benchmark
Alibaba N/A N/A
Amazon 
[{
	"label": "Sunglasses 1",
	"confidence": 0.996,
	"top": 463,
	"left": 435,
	"width": 110,
	"height": 35
}, {
	"label": "Person 1",
	"confidence": 0.995,
	"top": 408,
	"left": 89,
	"width": 760,
	"height": 434
}, {
	"label": "Person 2",
	"confidence": 0.886,
	"top": 586,
	"left": 19,
	"width": 53,
	"height": 126
}]
GeneXusAI - Amazon - DetectObjects
 11121ms
Baidu N/A N/A
Google 
[{
    "label": "Person",
    "confidence": 0.7952577,
    "top": 477,
    "left": 115,
    "width": 722,
    "height": 369
}, {
    "label": "Man",
    "confidence": 0.64812267,
    "top": 409,
    "left": 146,
    "width": 600,
    "height": 372
}]
GeneXusAI - Google - DetectObjects
 8218ms
IBM N/A N/A
Microsoft 
[{
    "label": "outdoor",
    "confidence": 0.996472358703613,
    "top": 0,
    "left": 0,
    "width": 0,
    "height": 0
}, {
    "label": "sky",
    "confidence": 0.980476856231689,
    "top": 0,
    "left": 0,
    "width": 0,
    "height": 0
}, {
    "label": "person",
    "confidence": 0.974854707717896,
    "top": 0,
    "left": 0,
    "width": 0,
    "height": 0
}, {
    "label": "man",
    "confidence": 0.922227799892426,
    "top": 0,
    "left": 0,
    "width": 0,
    "height": 0
}, {
    "label": "building",
    "confidence": 0.8099485039711,
    "top": 0,
    "left": 0,
    "width": 0,
    "height": 0
}]
4896ms
MLKit 
[{
    "label": "FASHION_GOOD",
    "confidence": 0.937,
    "top": 674,
    "left": 446,
    "width": 355,
    "height": 369
}, {
    "label": "FASHION_GOOD",
    "confidence": 0.71,
    "top": 724,
    "left": 369,
    "width": 55,
    "height": 90
}, {
    "label": "FASHION_GOOD",
    "confidence": 0.582,
    "top": 515,
    "left": 199,
    "width": 822,
    "height": 534
}, {
    "label": "UNKNOWN",
    "confidence": 0,
    "top": 716,
    "left": 186,
    "width": 48,
    "height": 122
}, {
    "label": "UNKNOWN",
    "confidence": 0,
    "top": 729,
    "left": 13,
    "width": 64,
    "height": 145
}]
999ms
SAP N/A N/A
Tencent 
[{
	"label": "GXAI_TCN_OBJECT_668",
	"confidence": 0.623,
	"top": 0,
	"left": 0,
	"width": 0,
	"height": 0
}, {
	"label": "GXAI_TCN_OBJECT_873",
	"confidence": 0.204,
	"top": 0,
	"left": 0,
	"width": 0,
	"height": 0
}, {
	"label": "GXAI_TCN_OBJECT_682",
	"confidence": 0.101,
	"top": 0,
	"left": 0,
	"width": 0,
	"height": 0
}, {
	"label": "GXAI_TCN_OBJECT_442",
	"confidence": 0.017,
	"top": 0,
	"left": 0,
	"width": 0,
	"height": 0
}, {
	"label": "GXAI_TCN_OBJECT_698",
	"confidence": 0.01,
	"top": 0,
	"left": 0,
	"width": 0,
	"height": 0
}]
5587ms

Notes

  • The label assigned for an object depends on the provider used. Additional information can be found on the OutputRegion.Info field if it is given by the provider.
  • Maximum image file size is 10MB.
  • Tencent AI returns labels as 'GXAI_TCN_OBJECT_{id}' tags, where the {id} is a numeric class provided by Tencent. You can use a Language object for mapping each tag with string label as other provider does. You can download this xpz which contains the SimplifiedChinese language object with the official mapping, and also contains the English language mapping using machine translation (it can be inaccurate). Once you import this xpz, for example, the 'GXAI_TCN_OBJECT_668' label on the sample section will be translated to '清真寺' (or 'mosque' in English) if you have set Translation type property in your environment with 'Run-time' value.
  • Rectangle area has zero values when the provider is not allowed to identify the region.
  • GeneXusAI does not provide support for drawing a rectangle over an image. This action is the responsibility of the developer.
    TIP: For Web applications, a good alternative can be combining HTML5 Canvas control with JavaScript with User Control object. On the other hand, for Smart Devices you could use Image Map control on which you can set the processed image as background and 'draw' square regions (i.e. set a border color on the table item of the grid).

Scope

Platforms Web(.NET,.NETCore,Java), SmartDevices(Android,iOS)
Connectivity Online

Availability

This procedure is available as of GeneXus 16.

  • As of GeneXus 16 upgrade 1:
    - Google Cloud AI is available.
  • As of GeneXus 16 upgrade 2:
    - Amazon WS and Tencent AI are available.
    - Google returns rectangles with values different from 0. Applies for .NET and Java environments, .NET Core is still limited.
  • As of GeneXus 16 upgrade 10:
    - Google returns rectangles with values different from 0 for .NET Core.
  • As of GeneXus 16 Upgrade 11:
    - Firebase ML Kit is available for Android