Enterprise AI
Table of contents
Official Content
  • This documentation is valid for:

The Evaluation Plan API allows you to manage and execute evaluation plans for AI Assistants. These plans define metrics and weights to evaluate the performance of an AI Assistant.

Each evaluation plan includes the Assistant to be evaluated, the dataset used and a list of system metrics with their respective weights. The metrics can be, for example, accuracy or fluency, and each has an identifier (Id) and a weight within the plan.

The API provides several endpoints to manage these plans, such as creating, updating, deleting and querying evaluation plans, as well as adding, updating or deleting metrics associated with each plan. In addition, all available metrics can be queried.

The following endpoints require a Globant Enterprise AI API token related to project scope.

Check the generic variables needed to use the API.

Endpoints

Method Path Description
GET /evaluationPlanApi/evaluationPlans Retrieves a list of all evaluation plans.
POST /evaluationPlanApi/evaluationPlan Creates a new evaluation plan.
GET /evaluationPlanApi/evaluationPlan/{evaluationPlanId} Retrieves a specific evaluation plan by ID.
PUT /evaluationPlanApi/evaluationPlan/{evaluationPlanId} Updates a specific evaluation plan by ID.
DELETE /evaluationPlanApi/evaluationPlan/{evaluationPlanId} Deletes a specific evaluation plan by ID.
GET /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetrics Retrieves system metrics associated with a specific evaluation plan.
POST /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric Adds a system metric to a specific evaluation plan.
GET /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId} Retrieves a specific system metric from an evaluation plan.
PUT /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId} Updates a specific system metric within an evaluation plan.
DELETE /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId} Deletes a specific system metric from an evaluation plan.
GET /evaluationPlanApi/systemMetrics Retrieves a list of all available system metrics.
GET /evaluationPlanApi/systemMetric/{systemMetricId} Retrieves a specific system metric by ID.
POST /evaluationPlanApi/evaluationPlan/{evaluationPlanId} Executes a specific evaluation plan.

GET/evaluationPlanApi/evaluationPlans

Retrieves a list of all evaluation plans.

Request

  • Method: GET
  • Path: /evaluationPlanApi/evaluationPlans
  • Request Body: Empty

Response

The response will be a JSON array containing objects, each representing an evaluation plan. The exact structure of the objects will depend on the implementation, but it will likely include fields such as evaluationPlanId, evaluationPlanName, evaluationPlanType, evaluationPlanAssistantId, evaluationPlanAssistantName, evaluationPlanAssistantRevision, evaluationPlanProfileName, dataSetId (optional), and a list of systemMetrics (each with systemMetricId and systemMetricWeight). Error information will be included if the request fails.

{
  "evaluationPlans": [
    {
      "evaluationPlanId": "UUID",
      "evaluationPlanName": "string",
      "evaluationPlanType": "string",
      "evaluationPlanAssistantId": "UUID",
      "evaluationPlanAssistantName": "string",
      "evaluationPlanAssistantRevision": "string",
      "evaluationPlanProfileName": "string",
      "dataSetId": "string",
      "systemMetrics": [
        {
          "systemMetricId": "string",
          "systemMetricWeight": 0.0
        }
      ]
    }
  ],
  "errors": [ ]
}

cURL Sample

curl -X GET "$BASE_URL/evaluationPlanApi/evaluationPlans" \
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" \
  -H "Accept: application/json"

POST/evaluationPlanApi/evaluationPlan

Creates a new evaluation plan.

Request

  • Method: POST
  • Path: /evaluationPlanApi/evaluationPlan

Request Body

A JSON object with the following structure:

{
  "evaluationPlanName": "string",  //Name of the evaluation plan
  "evaluationPlanType": "string",  //Type of assistant (e.g., "TextPromptAssistant", "RAG Assistant")
  "evaluationPlanAssistantId": "UUID", //ID of the assistant (optional, required for TextPromptAssistant)
  "evaluationPlanAssistantName": "string", //Name of the assistant (optional, required for TextPromptAssistant)
  "evaluationPlanAssistantRevision": "string", //Revision of the assistant (optional, required for TextPromptAssistant)
  "evaluationPlanProfileName": "string", //Name of the RAG profile (optional, required for RAG Assistant)
  "dataSetId": "string", //ID of the dataset (optional)
  "systemMetrics": [ //Array of system metrics
    {
      "systemMetricId": "string", //ID of the system metric
      "systemMetricWeight": 0.0 //Weight of the system metric (between 0 and 1)
    }
  ]
}

Response

A JSON object containing the newly created evaluation plan's data, including a newly generated evaluationPlanId. Error details will be included in the response body if the request fails. A typical successful response might look like this:

{
  "evaluationPlanId": "UUID",
  "evaluationPlanName": "string",
  "evaluationPlanType": "string",
  "evaluationPlanAssistantId": "UUID",
  "evaluationPlanAssistantName": "string",
  "evaluationPlanAssistantRevision": "string",
  "evaluationPlanProfileName": "string",
  "dataSetId": "string",
  "systemMetrics": [
    {
      "systemMetricId": "string",
      "systemMetricWeight": "number"
    }
  ],
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X POST "$BASE_URL/evaluationPlanApi/evaluationPlan" \
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
    "evaluationPlanName": "My New Plan",
    "evaluationPlanType": "TextPromptAssistant",
    "evaluationPlanAssistantId": "existing-assistant-UUID",
    "systemMetrics": [
      {"systemMetricId": "metricA", "systemMetricWeight": 0.6},
      {"systemMetricId": "metricB", "systemMetricWeight": 0.4}
    ]
  }'

GET/evaluationPlanApi/evaluationPlan/{evaluationPlanId}

Retrieves a specific evaluation plan using its ID.

GET /evaluationPlan/{evaluationPlanId}

This endpoint retrieves a specific evaluation plan using its ID.

Request

  • Method: GET
  • Path: /evaluationPlanApi/evaluationPlan/{evaluationPlanId}
  • Request Body: Empty

Response

A JSON object representing the requested evaluation plan. The structure will be similar to the one returned by a POST request, including fields like evaluationPlanId, evaluationPlanName, evaluationPlanType, evaluationPlanAssistantId, evaluationPlanAssistantName, evaluationPlanAssistantRevision, evaluationPlanProfileName, dataSetId, and systemMetrics. If the plan is not found or an error occurs, the response body will contain error details. Here is a successful response example:

{
  "evaluationPlanId": "UUID",
  "evaluationPlanName": "string",
  "evaluationPlanType": "string",
  "evaluationPlanAssistantId": "UUID",
  "evaluationPlanAssistantName": "string",
  "evaluationPlanAssistantRevision": "string",
  "evaluationPlanProfileName": "string",
  "dataSetId": "string",
  "systemMetrics": [
    {
      "systemMetricId": "string",
      "systemMetricWeight": "number"
    }
  ],
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X GET "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef" \
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" \
  -H "Accept: application/json"

PUT/evaluationPlanApi/evaluationPlan/{evaluationPlanId}

Updates an existing evaluation plan.

Request

  • Method: PUT
  • Path: evaluationPlanApi/evaluationPlan/{evaluationPlanId}

Request Body

A JSON object with the same structure as the request body for the POST /evaluationPlan endpoint. You can update any of the fields (name, type, assistant ID, profile name, dataset ID, system metrics). You must include the evaluationPlanId in the request body for the update to work correctly. Only the fields you provide in the request body will be updated; others will remain unchanged.

{
  "evaluationPlanId": "UUID",
  "evaluationPlanName": "string",
  "evaluationPlanType": "string",
  "evaluationPlanAssistantId": "UUID",
  "systemMetrics": [
    {
      "systemMetricId": "string",
      "systemMetricWeight": "number"
    }
  ]
}

Response

A JSON object representing the updated evaluation plan. This will include the updated fields and any unchanged fields from the original plan. Error details will be included if the request fails. Here is a successful response example:

{
  "evaluationPlanId": "UUID",
  "evaluationPlanName": "string",
  "evaluationPlanType": "string",
  "evaluationPlanAssistantId": "UUID",
  "evaluationPlanAssistantName": "string",
  "evaluationPlanAssistantRevision": "string",
  "evaluationPlanProfileName": "string",
  "dataSetId": "string",
  "systemMetrics": [
    {
      "systemMetricId": "string",
      "systemMetricWeight": "number"
    }
  ],
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X PUT "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json" 
  -H "Content-Type: application/json" 
  -d '{
    "evaluationPlanId": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
    "evaluationPlanName": "Updated Plan Name",
    "evaluationPlanType": "TextPromptAssistant",
    "evaluationPlanAssistantId": "new-assistant-UUID",
    "systemMetrics": [
      {"systemMetricId": "metricZ", "systemMetricWeight": 0.9}
    ]
  }'

DELETE/evaluationPlanApi/evaluationPlan/{evaluationPlanId}

Deletes a specific evaluation plan.

Request

  • Method: DELETE
  • Path: /evaluationPlanApi/evaluationPlan/{evaluationPlanId}
  • Request Body: Empty

Response

The response body will typically be empty or contain a minimal success message on successful deletion. If an error occurs, the response body will contain details about the error. A successful response might look like this (though an empty body is also common):

{
  "message": "Evaluation plan deleted successfully."
}

cURL Sample

curl -X DELETE "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json"

GET/evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetrics

Retrieves the system metrics associated with a specific evaluation plan.

Request

  • Method: GET
  • Path: /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetrics
  • Request Body: Empty

Response

A JSON object containing an array of system metrics associated with the specified evaluation plan. Each system metric object will likely contain at least systemMetricId and systemMetricWeight. Here is a successful response example:

{
  "systemMetrics": [
    {
      "systemMetricId": "string",
      "systemMetricWeight": "number"
    }
  ],
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X GET "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef/evaluationPlanSystemMetrics" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json"

POST/evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric

Adds a new system metric to an existing evaluation plan.

Request

  • Method: POST
  • Path: evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric

Request Body

A JSON object specifying the system metric to add:

{
  "systemMetricId": "string",
  "systemMetricWeight": "number"
}

Response

On success, the response body might contain the updated list of system metrics for the evaluation plan or a simple success message. If an error occurs, the response body will contain details about the error. Here is a successful response example:

{
  "systemMetrics": [
    {
      "systemMetricId": "string",
      "systemMetricWeight": "number"
    }
  ],
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X POST "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef/evaluationPlanSystemMetric" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json" 
  -H "Content-Type: application/json" 
  -d '{
    "systemMetricId": "metricC",
    "systemMetricWeight": 0.2
  }'

GET/evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId}

Retrieves a specific system metric from a given evaluation plan.

Request

  • Method: GET
  • Path: /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId}
  • Request Body: Empty

Response

A JSON object representing the requested system metric. It will at least contain systemMetricId and systemMetricWeight. Here is a successful response example:

{
  "systemMetricId": "string",
  "systemMetricWeight": "number",
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X GET "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef/evaluationPlanSystemMetric/metricB" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json"

PUT/evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId}

Updates a specific system metric within an existing evaluation plan.

Request

  • Method: PUT
  • Path:/evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId}

Request Body

A JSON object containing the updated systemMetricWeight. The systemMetricId should not be included in the request body; it is specified in the URL.

{
  "systemMetricWeight": "number"
}

Response

On success, the response might contain the updated system metric or a simple success message. If an error occurs, the response body will contain error details. Here is a successful response example:

{
  "systemMetricId": "string",
  "systemMetricWeight": "number",
  "errors": []
}

cURL Sample

curl -X PUT "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef/evaluationPlanSystemMetric/metricB" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json" 
  -H "Content-Type: application/json" 
  -d '{
    "systemMetricWeight": 0.7
  }'

DELETE/evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId}

Deletes a specific system metric from an evaluation plan.

Request

  • Method: DELETE
  • Path: /evaluationPlanApi/evaluationPlan/{evaluationPlanId}/evaluationPlanSystemMetric/{systemMetricId}
  • Request Body: Empty

Response

The response body will typically be empty or contain a minimal success message on successful deletion. If an error occurs, the response body will contain details about the error. A successful response might look like this (though an empty body is also common):

{
  "message": "System metric deleted successfully."
}

cURL Sample

curl -X DELETE "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef/evaluationPlanSystemMetric/metricB" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json"

GET/evaluationPlanApi/systemMetrics

Retrieves a list of all available system metrics that can be used in evaluation plans.

Request

  • Method: GET
  • Path: /evaluationPlanApi/systemMetrics
  • Request Body: Empty

Response

A JSON object containing an array of system metric objects. Each object will likely contain at least a systemMetricId and a systemMetricName (or similar descriptive field). Here is a successful response example:

{
  "systemMetrics": [
    {
      "systemMetricId": "string",
      "systemMetricName": "string"
    }
  ],
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X GET "$BASE_URL/evaluationPlanApi/systemMetrics" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json"

GET/evaluationPlanApi/systemMetric/{systemMetricId}

Retrieves a specific system metric using its ID.

Request

  • Method: GET
  • Path: /evaluationPlanApi/systemMetric/{systemMetricId}
  • Request Body: Empty

Response

A JSON object representing the requested system metric. The exact fields will depend on the implementation, but it will likely include at least systemMetricId and a descriptive field like systemMetricName or description. Here is a successful response example:

{
  "systemMetricId": "string",
  "systemMetricName": "string",
  "description": "string",
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X GET "$BASE_URL/evaluationPlanApi/systemMetric/metricB" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json"

POST/evaluationPlanApi/evaluationPlan/{evaluationPlanId}

Initiates the execution of a previously defined evaluation plan. The evaluation plan's configuration (assistant, dataset, metrics, and weights) determines how the assessment is performed. This is likely an asynchronous operation; the response will not contain the full evaluation results immediately.

Request

  • Method: POST
  • Path: /evaluationPlanApi/evaluationPlan/{evaluationPlanId}
  • Request Body: Empty

Response

The evaluation plan execution was successfully initiated. The response body will likely contain information to track the progress of the evaluation, such as an executionId.

{
  "executionId": "UUID",
  "status": "string",
  "message": "string",
  "errors": [
    {
      "code": "integer",
      "message": "string"
    }
  ]
}

cURL Sample

curl -X POST "$BASE_URL/evaluationPlanApi/evaluationPlan/a1b2c3d4-e5f6-7890-1234-567890abcdef" 
  -H "Authorization: Bearer $SAIA_PROJECT_APITOKEN" 
  -H "Accept: application/json"

See Also

How to evaluate an AI Assistant

—
Last update: 10 March 2025 by rp
Last update: March 2025 | © GeneXus. All rights reserved. GeneXus Powered by Globant