GeneXus Cognitive API - Analyze procedure

This documentation is valid for:
GeneXus 18 Help
GeneXus 17 Help
GeneXus 16 Help

Gets the analysis result of a video after it has been processed by GeneXus Cognitive API - Process procedure.

Parameters

in:&videoId :: VideoId, GeneXusAI.Video
The video identifier to be analyzed.
in:&provider :: Provider, GeneXusAI.Configuration
Provider settings.
inout:&Messages :: Messages, GeneXus.Common
A collection of warning and error messages returned by the task. You should check in your code if an error was returned. Refer to error codes and descriptions for more information.
out:&OutputAnalysis :: OutputAnalysis, GeneXusAI.Video
The result of the performed analysis.

Configuration

The following table resumes the configuration properties (access credentials) you must set in order to use this AI task.

	PropertyKey
ProviderType	Key	KeySecret	Account
Alibaba	内容安全 app-key	用户AccessKey	用户AccessKey
Amazon	-	-	-
Baidu	视频内容分析	视频内容分析
Google	Video Intelligence API	-	-
IBM	-	-	-
Microsoft	Video Indexer	-	Azure Connect
SAP	-	-	-
Tencent	-	-	-

Sample

The available features by each provider are listed in the following table. In case there is no result for some feature, its field will be empty.

Feature	Alibaba	Amazon	Baidu	Google	IBM	Microsoft	SAP	Tencent
Speeches	-	-	✓	✓	-	✓	-	-
Categories	✓	-	✓	✓	-	✓	-	-
Faces	✓	-	✓	-	-	✓	-	-
OCRs	-	-	✓	-	-	✓	-	-
Sentiments	-	-	-	-	-	✓	-	-

Taking the following video input, the table below shows the analysis made for each provider and the time elapsed between it was sent to be processed until it has been ready for being analyzed.

GeneXusAI - Video module - Sample

0:00/0:00

Provider	Output	Benchmark
Alibaba	{ "Completed": true, "VideoId": "{alibaba_video_id}", "Progress": 100.00, "Speeches": [], "Categories": [{ "label": "iPod", "confidence": 0.941, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:22.000" }] }, { "label": "bending metal", "confidence": 0.697, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:22.000" }] }, { "label": "binder", "confidence": 0.628, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:22.000" }] }, { "label": "microwave", "confidence": 0.471, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:22.000" }] }, { "label": "unboxing", "confidence": 0.416, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:22.000" }] }], "Sentiments": [], "Faces": [], "OCRs": [] }	-
Amazon	-	-
Baidu	{ "Completed": true, "VideoId": "{baidu_video_id}", "Progress": 100.00, "Speeches": [{ "label": "就是个球儿的，", "confidence": 1.000, "Info": [{ "property": "START", "value": "00:00:00.000" }, { "property": "END", "value": "00:00:05.000" }] }, { "label": "今天，", "confidence": 1.000, "Info": [{ "property": "START", "value": "00:00:06.000" }, { "property": "END", "value": "00:00:11.000" }] }, { "label": "二点，", "confidence": 1.000, "Info": [{ "property": "START", "value": "00:00:11.000" }, { "property": "END", "value": "00:00:15.000" }] }], "Categories": [{ "label": "手机", "confidence": 0.953, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { ... },{ "property": "START-4", "value": "00:00:08.000" }] }, { "label": "黑屏", "confidence": 0.898, "Info": [{ "property": "START-1", "value": "00:00:10.000" }, { ... }, { "property": "START-8", "value": "00:00:17.000" }] }, { "label": "空白纸", "confidence": 0.868, "Info": [{ "property": "START-1", "value": "00:00:09.000" }] }, { "label": "书架", "confidence": 0.743, "Info": [{ "property": "START-1", "value": "00:00:02.000" }] }, { "label": "荧光字", "confidence": 0.642, "Info": [{ "property": "START-1", "value": "00:00:18.000" }, { ... }, { "property": "START-5", "value": "00:00:22.000" }] }, { "label": "模糊无意义图片", "confidence": 0.483, "Info": [{ "property": "START-1", "value": "00:00:05.000" }] }, { "label": "冰箱", "confidence": 0.322, "Info": [{ "property": "START-1", "value": "00:00:03.000" }, { "property": "START-2", "value": "00:00:04.000" }] }, { "label": "剪刀", "confidence": 0.255, "Info": [{ "property": "START-1", "value": "00:00:06.000" }] }], "OCRs": [{ "label": "Dream like a kid", "confidence": 1, "top": 334, "left": 471, "width": 285, "height": 39, "Info": [{ "property": "START-1", "value": "00:00:10.000" }] }, { "label": "Work like an artist", "confidence": 0.995, "top": 333, "left": 447, "width": 354, "height": 40, "Info": [{ "property": "START-1", "value": "00:00:11.000" }, { "property": "START-2", "value": "00:00:12.000" }] }, { "label": "Create like a pro", "confidence": 0.98, "top": 334, "left": 475, "width": 292, "height": 40, "Info": [{ "property": "START-1", "value": "00:00:13.000" }] }, { "label": "Create like a pro", "confidence": 0.99, "top": 334, "left": 475, "width": 292, "height": 48, "Info": [{ "property": "START-1", "value": "00:00:14.000" }, { ... }, { "property": "START-4", "value": "00:00:17.000" }] }, { "label": "GeneXus", "confidence": 0.96, "top": 318, "left": 437, "width": 369, "height": 85, "Info": [{ "property": "START-1", "value": "00:00:18.000" }] }, { "label": "GeneXus Simplify Software Development", "confidence": 0.98, "top": 319, "left": 347, "width": 558, "height": 147, "Info": [{ "property": "START-1", "value": "00:00:19.000" }, { ... }, { "property": "START-3", "value": "00:00:21.000" }] }, { "label": "TM", "confidence": 0.99, "top": 323, "left": 792, "width": 29, "height": 19, "Info": [{ "property": "START-1", "value": "00:00:19.000" }, { ... }, { "property": "START-3", "value": "00:00:21.000" }] }, { "label": "Start creating now", "confidence": 0.97, "top": 321, "left": 458, "width": 327, "height": 45, "Info": [{ "property": "START-1", "value": "00:00:22.000" }] }] }	18478ms
Google	{ "Completed": true, "Progress": 100, "Id": "{google_video_id}", "Speeches": [{ "label": "Just like a true artist a masterpiece of Vols as it happens in real time.", "confidence": 0.8241047, "Info": [{ "property": "START", "value": "0s" }, { "property": "END", "value": "4.200s" } ] }, { "label": " Genexus with live editing dream like a kid work like an artist create like a pro.", "confidence": 0.8710277, "Info": [{ "property": "START", "value": "7.700s" }, { "property": "END", "value": "14.700s" } ] } ], "Categories": [{ "label": "mobile phone", "confidence": 0.4763316, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:02.021" } ] }, { "label": "mobile device", "confidence": 0.67164063, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:02.021" } ] }, { "label": "smartphone", "confidence": 0.36822546, "Info": [{ "property": "START-1", "value": "00:00:00.000" }, { "property": "END-1", "value": "00:00:02.021" } ] }, { "label": "feature phone", "confidence": 0.7129101571428571428571428571, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-7", "value": "00:00:05.080" } ] }, { "label": "black", "confidence": 0.95241495, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "product", "confidence": 0.7823901175, "Info": [{ "property": "START-1", "value": "00:00:07.096" }, { "property": "START-2", "value": "00:00:17.069" }, { ... }, { "property": "START-4", "value": "00:00:21.009" } ] }, { "label": "gadget", "confidence": 0.92548106375, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-8", "value": "00:00:07.096" } ] }, { "label": "electronics", "confidence": 0.7306240733333333333333333333, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-9", "value": "00:00:09.014" } ] }, { "label": "brand", "confidence": 0.5673828673684210526315789474, "Info": [{ "property": "START-1", "value": "00:00:01.058" }, { "property": "START-2", "value": "00:00:02.060" }, { ... }, { "property": "START-19", "value": "00:00:21.009" } ] }, { "label": "communication device", "confidence": 0.7557767875, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-8", "value": "00:00:07.096" } ] }, { "label": "sky", "confidence": 0.728198775, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "flooring", "confidence": 0.41395074, "Info": [{ "property": "START-1", "value": "00:00:02.060" }, { "property": "START-2", "value": "00:00:03.063" } ] }, { "label": "shelf", "confidence": 0.4387352, "Info": [{ "property": "START-1", "value": "00:00:02.060" } ] }, { "label": "electronic device", "confidence": 0.8647810466666666666666666667, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-9", "value": "00:00:09.014" } ] }, { "label": "product design", "confidence": 0.6257422770588235294117647059, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-17", "value": "00:00:21.009" } ] }, { "label": "portable communications device", "confidence": 0.6782316714285714285714285714, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-7", "value": "00:00:05.080" } ] }, { "label": "shelving", "confidence": 0.412489015, "Info": [{ "property": "START-1", "value": "00:00:02.060" }, { "property": "START-2", "value": "00:00:04.069" } ] }, { "label": "computer hardware", "confidence": 0.44277436, "Info": [{ "property": "START-1", "value": "00:00:01.058" } ] }, { "label": "electronic instrument", "confidence": 0.49085233, "Info": [{ "property": "START-1", "value": "00:00:01.058" } ] }, { "label": "multimedia", "confidence": 0.4773627869230769230769230769, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-13", "value": "00:00:14.083" } ] }, { "label": "computer monitor", "confidence": 0.50538915, "Info": [{ "property": "START-1", "value": "00:00:07.096" } ] }, { "label": "phenomenon", "confidence": 0.7547649, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "darkness", "confidence": 0.702858785, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "mobile phone", "confidence": 0.8544927925, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-8", "value": "00:00:07.096" } ] }, { "label": "display device", "confidence": 0.66004828, "Info": [{ "property": "START-1", "value": "00:00:01.058" }, { "property": "START-2", "value": "00:00:07.096" } ] }, { "label": "graphics", "confidence": 0.57838248, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "midnight", "confidence": 0.490563215, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "document", "confidence": 0.5087058, "Info": [{ "property": "START-1", "value": "00:00:09.014" } ] }, { "label": "angle", "confidence": 0.55732546, "Info": [{ "property": "START-1", "value": "00:00:05.080" }, { "property": "START-2", "value": "00:00:06.087" }, { ... }, { "property": "START-5", "value": "00:00:10.030" } ] }, { "label": "screenshot", "confidence": 0.451265465, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "text", "confidence": 0.863238075, "Info": [{ "property": "START-1", "value": "00:00:06.087" }, { "property": "START-2", "value": "00:00:07.096" }, { ... }, { "property": "START-14", "value": "00:00:21.009" } ] }, { "label": "atmosphere of earth", "confidence": 0.79306614, "Info": [{ "property": "START-1", "value": "00:00:12.064" } ] }, { "label": "graphic design", "confidence": 0.48747425, "Info": [{ "property": "START-1", "value": "00:00:17.069" }, { "property": "START-2", "value": "00:00:19.099" } ] }, { "label": "telephone", "confidence": 0.5075905257142857142857142857, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-7", "value": "00:00:05.080" } ] }, { "label": "telephony", "confidence": 0.5244649657142857142857142857, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-7", "value": "00:00:05.080" } ] }, { "label": "media", "confidence": 0.407669325, "Info": [{ "property": "START-1", "value": "00:00:03.063" }, { "property": "START-2", "value": "00:00:07.096" } ] }, { "label": "paper product", "confidence": 0.40085325, "Info": [{ "property": "START-1", "value": "00:00:06.087" }, { ... }, { "property": "START-4", "value": "00:00:10.030" } ] }, { "label": "computer accessory", "confidence": 0.41228032, "Info": [{ "property": "START-1", "value": "00:00:07.096" } ] }, { "label": "technology", "confidence": 0.8440843188888888888888888889, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-9", "value": "00:00:09.014" } ] }, { "label": "white", "confidence": 0.9594464, "Info": [{ "property": "START-1", "value": "00:00:12.064" } ] }, { "label": "paper", "confidence": 0.59338312, "Info": [{ "property": "START-1", "value": "00:00:05.080" }, { "property": "START-2", "value": "00:00:06.087" }, { ... }, { "property": "START-5", "value": "00:00:10.030" } ] }, { "label": "design", "confidence": 0.704395194, "Info": [{ "property": "START-1", "value": "00:00:05.080" }, { "property": "START-2", "value": "00:00:06.087" }, { ... }, { "property": "START-5", "value": "00:00:10.030" } ] }, { "label": "cellular network", "confidence": 0.69048995, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-7", "value": "00:00:05.080" } ] }, { "label": "computer wallpaper", "confidence": 0.6304926405882352941176470588, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-17", "value": "00:00:21.009" } ] }, { "label": "logo", "confidence": 0.715734012, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] }, { "label": "trademark", "confidence": 0.415102925, "Info": [{ "property": "START-1", "value": "00:00:19.099" }, { "property": "START-2", "value": "00:00:21.009" } ] }, { "label": "atmosphere", "confidence": 0.76842415, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-5", "value": "00:00:14.083" } ] }, { "label": "triangle", "confidence": 0.7155078, "Info": [{ "property": "START-1", "value": "00:00:05.080" } ] }, { "label": "light", "confidence": 0.8200014, "Info": [{ "property": "START-1", "value": "00:00:11.048" } ] }, { "label": "material property", "confidence": 0.44692206, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-8", "value": "00:00:06.087" } ] }, { "label": "smartphone", "confidence": 0.6743124275, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-8", "value": "00:00:07.096" } ] }, { "label": "product", "confidence": 0.528562268125, "Info": [{ "property": "START-1", "value": "00:00:00.020" }, { "property": "START-2", "value": "00:00:00.022" }, { ... }, { "property": "START-16", "value": "00:00:21.009" } ] }, { "label": "font", "confidence": 0.6727489471428571428571428571, "Info": [{ "property": "START-1", "value": "00:00:06.087" }, { "property": "START-2", "value": "00:00:07.096" }, { ... }, { "property": "START-14", "value": "00:00:21.009" } ] }, { "label": "space", "confidence": 0.434149805, "Info": [{ "property": "START-1", "value": "00:00:11.048" }, { "property": "START-2", "value": "00:00:12.020" }, { ... }, { "property": "START-10", "value": "00:00:21.009" } ] } ] }	12165ms
IBM	-	-
Microsoft	{ "Completed": true, "Progress": 100, "Id": "{microsoft_video_id}", "Speeches": [{ "label": "Just like a true artist a masterpiece evolved as it happens in real time, Genexus with live editing dream like a kid.", "confidence": 0.7632, "Info": [{ "property": "START-1", "value": "0:00:00" }, { "property": "END-1", "value": "0:00:11.41" }, { "property": "SPEAKER", "value": "#1" } ] }, { "label": "Work like an artist create like a pro.", "confidence": 0.7116, "Info": [{ "property": "START-1", "value": "0:00:11.42" }, { "property": "END-1", "value": "0:00:19.47" }, { "property": "SPEAKER", "value": "#1" } ] } ], "Categories": [{ "label": "indoor", "confidence": 0.9575, "Info": [{ "property": "START-1", "value": "0:00:00" }, { "property": "END-1", "value": "0:00:10.24" } ] }, { "label": "sign", "confidence": 0.8783, "Info": [{ "property": "START-1", "value": "0:00:17.92" }, { "property": "END-1", "value": "0:00:19.2" } ] } ], "Sentiments": [{ "score": 0.7003, "Info": [{ "property": "START-1", "value": "0:00:03.81" }, { "property": "END-1", "value": "0:00:05.91" }, { ... }, { "property": "END-3", "value": "0:00:19.05" } ] }, { "score": 0.9232, "Info": [{ "property": "START-1", "value": "0:00:00" }, { "property": "END-1", "value": "0:00:03.73" }, { ... }, { "property": "END-2", "value": "0:00:12.15" } ] } ], "OCRs": [{ "label": "Dream like a kid", "confidence": 0.9545, "top": 0, "left": 0, "width": 0, "height": 0, "Info": [{ "property": "START-1", "value": "0:00:10.08" }, { "property": "END-1", "value": "0:00:11.52" } ] }, { "label": "Work like an artist", "confidence": 0.9578, "top": 0, "left": 0, "width": 0, "height": 0, "Info": [{ "property": "START-1", "value": "0:00:11.52" }, { "property": "END-1", "value": "0:00:13.44" } ] }, { "label": "Create like a pro", "confidence": 0.9871, "top": 0, "left": 0, "width": 0, "height": 0, "Info": [{ "property": "START-1", "value": "0:00:13.44" }, { "property": "END-1", "value": "0:00:17.76" } ] }, { "label": "GeneD(us", "confidence": 0.436, "top": 0, "left": 0, "width": 0, "height": 0, "Info": [{ "property": "START-1", "value": "0:00:18.72" }, { "property": "END-1", "value": "0:00:19.2" } ] }, { "label": "Simplify Software Development", "confidence": 0.9214, "top": 0, "left": 0, "width": 0, "height": 0, "Info": [{ "property": "START-1", "value": "0:00:19.2" }, { "property": "END-1", "value": "0:00:22.08" } ] }, { "label": "Start creating now", "confidence": 0.991, "top": 0, "left": 0, "width": 0, "height": 0, "Info": [{ "property": "START-1", "value": "0:00:22.08" }, { "property": "END-1", "value": "0:00:22.2" } ] }, { "label": "genexus.com/genexusl 5", "confidence": 0.7955, "top": 0, "left": 0, "width": 0, "height": 0, "Info": [{ "property": "START-1", "value": "0:00:22.08" }, { "property": "END-1", "value": "0:00:22.2" } ] } ] }	11860ms
SAP	-	-
Tencent	-	-

Notes

The &videoId value is volatile. That's means nothing guarantee you that multiple invocations to this task with the same value for &videoId will retrieve a result. So, it is highly recommended you save the result after the first successful invocation.
Microsoft Cognitive Services:
- To get the Key/Account, please refer to this document.
- Despite no region is filled up on the 'Faces' field items (top, left, width, height fields are zero -- or empty), you will have a property with 'thumbnail' key whose value is a cropped image of the identified face.
Baidu AI only recognizes Chinese speech (or, at least, an approximation of it).

Scope

Generators:	.NET, Java, Android, Apple, Angular
Connectivity:	Online

Availability

This procedure is available as of GeneXus 16 upgrade 1.

As of GeneXus 16 upgrade 3:
- Baidu AI is available.
As of GeneXus 16 upgrade 4:
- Alibaba AI is available.