GeneXus SDK for SAP Leonardo: Using a pretrained ML service

Official Content

Calling SAP Leonardo ImageClassification Service from a GeneXus application.

In this first sample, we will work using the Pretrained SAP Services (SAP API Business Hub) with the security approach through APIKey.

For this reason, only for the service published here, GeneXus SDK for SAP Leonardo, has a Domain, called Settings, where the developer can storage two values:

Base URL for SAP Leonardo Machine Learning Services, initialized with ‘https://sandbox.api.sap.com/ml/’

APIKey, where the developer sets the value corresponding to the user, for test purposes.

image_201911511272_1_png

-Download and Import SDK Module

Start by including the SDK module in your Knowledge Base, to do so you can follow the steps in this article

-Creating a Transaction.

Create a GeneXus Transaction object by right-clicking on the Root Module and selecting New->Transaction

image_2018122116752_1_png

Name the transaction object as 'ImageClassification' and click Create.

The Transaction will be displayed in the main panel.

i5-EmptyTRN_png

Since in this particular case you will store images to classify, you are going to describe three main attributes:

  • Id – Unique key to identify a record, set as a numeric type. As default, the Numeric(4) type is assigned.
  • Description- The description assigned to the image, set as a variable character field VarChar type.
  • Image- The image itself. For this type of data, GeneXus has the Image type.

We can observe that GeneXus in most cases infers the data type according to the name of the attribute.

If the developer presses the dot character (.) while having the prompt in an attribute name line, GeneXus will add the name of the transaction. Therefore the developer will only need to enter the rest of the attribute name.

(Naming attributes in a unique way throughout the Knowledge Base is a highly recommended practice since GeneXus infers through the attribute name, valuable information that saves a lot of development time)

 

After describing the attributes, the transaction should look like this.

i6-FullTRN_png

The asterisk shows that the objects have unsaved changes. Once the transaction changes are saved, the asterisk character at the end will disappear.

Now save your work by clicking the diskette Icon or pressing Ctrl+S.

 

- Applying Fiori Pattern in a GeneXus Object

To apply the pattern, go to the 'Patterns' selector in the transaction object and click on Initialize Fiori.
i7-InitializeFiori_png

Applying Fiori Pattern for the first time in the Knowledge Base involves the automatic generation the FioriBaseObjects Module.

image_20181221171253_1_png

FioriBaseObjects Module is now included in our Knowledge Base. From this point on, you only have to choose which Fiori floorplan want to apply to your GeneXus Object.

i9-FioriBaseObjects_201913112338_1_png

To select a floorplan for a GeneXus Object, go to 'Patterns' selector and click 'Select Floorplan'.

i10-SelectFloorplan_201913112346_1_png

A list of different floorplans is shown. Select List Report floorplan.

i11-SelectListFiori_201913133115_1_png

Now save your work by clicking the diskette Icon or pressing Ctrl+S to apply the floorplan.

i12-ListReportApplied_png

-Create Classify button

User actions can be created in the ListReport as buttons.

Right click on 'Grid' element in the right panel and select New->User Action.
i13-AddUserAction_png

Now, set the Name property to "Classify" and the Caption property to "Classify" . Now you can see the button at the end of the List Report registry.

i14-ClassifyProperties_png

-Creating Web Panel (with Image and Grid)

To create a new Web Panel right click on the Root Module and select New->Web Panel

i15-NewWebPanel_png

Name the Web Panel 'ViewClassificationSample’ and click Create.

i16-WebPanelName_201913112952_1_png

Now save your work by clicking the diskette Icon or pressing Ctrl+S. The web panel now will appear in your KB explorer.

There are four main elements that we want to show in this web form.

  1. The image selected to process.
  2. A Grid object to list the following elements (3 & 4).
  3. The categories in which the service has classified the image.
  4. The matching percentage for each category.

1- To add an Image drag the Attribute/Variable control from the toolbox to the Web Panel.

Upon dragging the control, you can observe a window panel showing all attributes and variables defined in our Knowledge Base.

In this case, you will need to create a new variable, so click New Variable.

i17-NewVariable_png

Now, name the Variable 'ProcessedImage'. GeneXus will automatically infer the variable type since the variable ends in Image. Click OK.

i18-ImageVariable_201913113036_1_png

The Web Panel will look like this.

i20-ImageInWebPanel_png

Make sure that the Label Position property is set to ‘None’.

Set the AutoResize property to ‘False’, so the image has a fixed size.

Set Width property to ‘450px’.

i21-ImageProperties_png

2- Add a Grid control dragging the item from the toolbox to the bottom of Web Form.

i22-AddingGrid_png

3-Click New Variable to create a variable to be displayed in the Grid.

i23-NewGridVariable_png

To specify the Category variable, name the variable as ‘Category’ and select ‘Varchar’ as Data Type, with Maximum length assigned to ‘100’. When completed click Ok.

i24-Categoryvariableproperties_png

Once the variable is added the Web Panel will look like this

image_20181226113332_1_png

4-Now add a second Variable to the Grid, dragging the Attribute/Variable control to the right side of the Grid control.

i25-AddingSecondGridVariable_20191311363_1_png

Once the control is placed, click on ‘New Variable’ after the windows form appears.

i26-AddNew2ndVariable_png

Name the variable ‘MatchPercentage’ and assign the Data Type property to ‘Numeric’ with Length property set to ’10’ and Decimals property set to ‘5’.

Click Ok.

i26_5-MatchPercentageVariableProperties_png

Now your Web Panel will look like this.

i27-WebPanelCompleted_png

Now save your work by clicking the diskette Icon or pressing Ctrl+S.

As this panel will work with the Image Id (ImageClassificationId), it is required to define the input parameters to call this object.

To do this, we have to select the ‘Rules’ tab next to the WebForm tab.

In this tab, we can specify rules to this particular object.

image_201913115318_1_png

To define the input in the rules editor, specify the following line.

parm(in:ImageClassificationId);

parm is the rule that defines the input and output parameters; they can be in for input, out for output or inout for input/output.

Save your web panel pressing Ctrl+S or clicking the diskette icon.

-Link Classify button to Web Panel.

To link the Classify button to the Web Panel, click on the transaction and select the User Action added. Click the GX Object property to bind the Web Panel.

i28-LinkingClassify_png

All GeneXus objects will be displayed; you may want to filter using Pattern textbox.

Once you have found the ViewClassificationSample WebPanel, select OK.

i29-LinkingWebPanel_png

As the web panel object to be called has the input parameter of ImageClassificationId, you have to assign it to the User Action.

First, you need to define that the User Action sends a set of parameters.

Right click on User Action->Add->Parameters.

i30-AddParameter_png

You just the defined that the user action control sends a set of parameters, but you need to specify which are those parameters.

Right Click on Parameters->Add->Parameter.

i31-AddingAParameter_png

After adding the parameter, you have to set the name of the parameter in the properties.

i32-NamingParameter_2019131208_1_png

Name it ‘ImageClassificationId’

After naming the parameter, save by clicking the diskette icon or press Ctrl+S.

i33-ParameterNamed_png

-Code Service call upon load event of the web panel

The final step is to program the SAP Leonardo Machine Learning service call to process the image and work with the results on the web panel.

In the 'ViewClassificationSample' web panel, select the 'Events' Tab and from the dropdown list, click on Load.

i34-LoadEvent_png

Once selected the following code will appear denoting the beginning and the end of the Load Event block.

Event Load

Endevent

Inside of the event Load we will write this code.

 

     Event Load
     for each           
             &ProcessedImage = ImageClassificationImage
             &ImageClassificationBlob = ImageClassificationImage
       endfor

       &ImageClassificationFile.Source = &ImageClassificationBlob

      //Call SAP Leonardo API for Image Classification
      &IsResultOK = GeneXusSAPLeonardo.ImageClassification.POSTInferenceSync(&ImageClassificationFile,'','', &ResponseOkOUT, &ResponseErrorOUT, &ErrorOUT, &Message)

      if &IsResultOK
          for &ClassificationResults in &ResponseOkOUT.predictions
              for &Classification in &ClassificationResults.results
                  &Category = &Classification.label                         
                  &MatchPercentage = Round(&Classification.score * 100, 2 ) 
                  load
              endfor
          endfor
      endif
Endevent

 

What this code does, it will be reviewed line by line.

Before reviewing the code, define the variables and its types.

Variable Name

Data Type

ImageClassificationBlob

Blob

ImageClassificationFile

File

IsResultOK

Boolean

ResponseOkOUT

ResponseOk, GeneXusSAPLeonardo.ImageClassification

ResponseErrorOUT

ResponseError, GeneXusSAPLeonardo.ImageClassification

ErrorOUT

Error, GeneXusSAPLeonardo.ImageClassification

Message

Messages.Message, GeneXus.Common

ClassificationResults

ClassificationResult, GeneXusSAPLeonardo.ImageClassification

Classification

Classification, GeneXusSAPLeonardo.ImageClassification

It is important that the data type of a variable refers to the appropriate module, in this case as you are using the Image Classification service, the module is GeneXusSAPLeonardo.ImageClassification.

Take a look at what this code does.

 for each           
    &ProcessedImage = ImageClassificationImage
    &ImageClassificationBlob = ImageClassificationImage
 endfor

The For Each command performs a database query filtering registers where the registry id (defined in the Transaction object) is the same as the parameter provided id, once the Id matches, the register image data will be assigned to the image variable.

The registry image data will also be assigned to a blob type variable; this will be of use for the next line.

&ImageClassificationFile.Source = &ImageClassificationBlob

The image retrieved from the database is assigned as the source of the file to be processed by the Image Classification service.

Now you have all the required data to call the service.

&IsResultOK = GeneXusSAPLeonardo.ImageClassification.POSTInferenceSync(&ImageClassificationFile, '', '', &ResponseOkOUT, &ResponseErrorOUT, &ErrorOUT, &Message)

The service call returns a Boolean indicating whether the service executed successfully or not.

The POSTInferenceSync method is included in GXSAPLeonardo.ImageClassification module and has the following parameters.

Parameter

In/ Out

Data Type

Value passed

&file

In

File

&ImageClassificationFile

&options

In

Varchar(256)

‘ ’

&Oauth2_ClientCredentials

In

Character(20)

‘ ‘

&ResponseOkOUT

Out

ResponseOk, GeneXusSAPLeonardo.ImageClassification

&ResponseOkOUT

&ResponseErrorOUT

Out

ResponseError, GeneXusSAPLeonardo.ImageClassification

&ResponseErrorOUT

&ErrorOUT

Out

Error, GeneXusSAPLeonardo.ImageClassification

&ErrorOUT

&HttpMessage

Out

Messages.Message, GeneXus.Common

&Message

&IsSuccess

Out

Boolean

&ResultOK

The service call will load the elements in the structure of the ImageClassification Service objects.

To display the results, we have to read the information loaded in those objects as a result of the service call.

if &IsResultOK
   for &ClassificationResults in &ResponseOkOUT.predictions
       for &Classification in &ClassificationResults.results
           &Category = &Classification.label                                      
           &MatchPercentage = Round(&Classification.score * 100, 2 )              
           load
        endfor
    endfor
endif

If the procedure executed without errors you iterate through every Classification results in the response (In this case, since you called the service for one image, there will be only one item, but the service can process more than one image in one call, this is due that the call allows .zip files as file parameters).

For each classification in the Classification Result, the name of the category in which the object was classified and its matching score to that category is loaded in the grid (using the load command).

Now you are ready to Build and Run your application. Press F5 or click Run image_2018122715461_1_png

This is the main menu of the application. Click on the Image Classification Sample tile to access the Fiori List Report.

i35-MainMenu_png

The List Report shows no elements. Click Insert to add a new element.

i36-EmptyListReport_png

To add a new element enter a unique Id: '0'.

Choose a name for the image: 'Puppy'.

Select the image clicking the Change button.

i37-AddImage_png

After browsing for the image select Confirm.

image_2018122716256_1_png

Now that you have selected the image click on confirm to create the registry.

i39-CreateRecord_png

Once the image is listed, click on 'Classify' to call SAP Image Classification service.

i40-ClassifyImage_png

This is the result.

The image displays the service call results, classifing it acording to default categories inclueded in the pretrained model of the Image Classification service available in SAP API Business Hub.

Was this page helpful?
What Is This?
Your feedback about this content is important. Let us know what you think.