Scheduled Jobs

Jobs

The Jobs resource represents a job executed asynchronously. Jobs are used to perform long-running tasks that can be monitored and managed through the API. Jobs can be created, updated, and deleted, and their status can be monitored and queried.

Operations

Job Results

The Job Results resource represents the results of a job that has been executed. Job results can be created, updated, and deleted, and their status can be monitored and queried.

Operations

Processing logs

The Job Results resource represents the results of data monitors that has been executed. Processing log results can be created, updated, and deleted, and their status can be monitored and queried.

Operations

Scheduled Jobs

Operations

Create Scheduled Job

Request

Create a new scheduled job.

Bodyapplication/jsonrequired

scheduledJobobject(ScheduledJobCreate)required

ScheduledJob create model.

scheduledJob.namestring(JobName)

Name of a Job.

Example: "Process vendor data."

scheduledJob.descriptionstring(JobDescription)

Detailed description of a Job.

Example: "I started this job to improve quality of our data."

scheduledJob.organizationIdstring(OrganizationId)

Uniquely identifying ID of the organization.

Example: "cdq_monitor"

scheduledJob.workspaceIdstring(WorkspaceId)

Uniquely identifying ID of the workspace.

Example: "c074b9f3-abf0-4f8e-9a20-74deb6cfa2a4"

scheduledJob.clientIdstring(ClientId)

Uniquely identifying ID of the client.

Example: "12345678-1234-5678-1234-567812345678"

scheduledJob.typestring(JobTypeParam)

The type of the job.

Enum Value	Description
AML_MAINTENANCE_JOB	AML maintenance job
BANK_DOCUMENT_DATA_INGESTION_JOB	Bank document data ingestion job
BANK_TABULAR_DATA_INGESTION_JOB	Bank tabular data ingestion job
BANK_TREE_DATA_INGESTION_JOB	Bank tree data ingestion job
BANKACCOUNT_DATA_CONFIRMATION_JOB	Bankaccount data confirmation job
BANKACCOUNT_DATA_IMPORT_JOB	Bankaccount data import job
BANKACCOUNT_DATA_STATISTICS_JOB	Bankaccount data statistics job
BANKACCOUNT_DATA_WHITELIST_CURATION_JOB	Bankaccount data whitelist curation job
BANKACCOUNT_DATA_WHITELIST_IMPORT_JOB	Bankaccount data whitelist import job
BATCH_LOOKUP_JOB	Batch lookup job

Example: "BANKACCOUNT_DATA_CONFIRMATION_JOB"

scheduledJob.dockerImageUrlstring(JobDockerImageUrl)non-empty

The URL of the Docker image.

Example: "docker-image-url"

scheduledJob.resourceRequestobject(ResourceRequest)

Resource request.

scheduledJob.environmentVariablesArray of objects(EnvironmentVariable)<= 100 items

Max 512 kB allowed.

scheduledJob.labelsArray of objects(Label)<= 50 items

List of labels.

scheduledJob.volumesArray of objects(Volume)<= 50 items

List of volumes.

scheduledJob.locksArray of objects(Locks)<= 5 items

Enables job locking. Prevents from running duplicated jobs of the same locks set.

Checks against NOT FINISHED, CANCELED nor FAILED jobs.

When tag=DATA_MONITOR is provided together with locks, then job is locked only against another DATA_MONITOR jobs.

Example: [ {"key": "jobType", "value": "DATA_CURATION_JOB"}, {"key": "storageId", "value": "87621345678976543245678965432"}, {"key": "dataSourceIds", "value": hash(["dataSourceId1", "dataSourceId2", "dataSourceId3", "dataSourceId4", "dataSourceId5", "dataSourceId6"])} ]

featuresOnArray of strings(ScheduledJobCreateFeatureParam)

List of features to be activated.

Items Value	Description
USE_ACCESS_FROM_REQUEST	Fills organizationId and workspaceID from the request body. If not provided, values are taken from JWT.

Example: ["USE_ACCESS_FROM_REQUEST"]

curl -i -X POST \
  https://idp.cdq.com/_mock/apis/jobs-management-api/api-v2/v2/scheduledjobs \
  -H 'Content-Type: application/json' \
  -d '{
    "scheduledJob": {
      "name": "Process vendor data.",
      "description": "I started this job to improve quality of our data.",
      "organizationId": "cdq_monitor",
      "workspaceId": "c074b9f3-abf0-4f8e-9a20-74deb6cfa2a4",
      "clientId": "12345678-1234-5678-1234-567812345678",
      "type": "BANKACCOUNT_DATA_CONFIRMATION_JOB",
      "dockerImageUrl": "docker-image-url",
      "resourceRequest": {
        "memoryLimit": 1024,
        "memoryRequest": 1024,
        "cpuLimit": 0.2,
        "cpuRequest": 0.1,
        "xmxLimit": 512
      },
      "environmentVariables": [
        {
          "key": "STORAGE_ID",
          "value": "123456789"
        }
      ],
      "labels": [
        {
          "key": "my label",
          "value": "my job"
        }
      ],
      "volumes": [
        {
          "mountPath": "/path",
          "subPath": "subpath"
        }
      ],
      "locks": [
        {
          "key": "jobType",
          "value": "SUBSCRIPTION_FAT_JOB"
        }
      ]
    },
    "featuresOn": [
      "USE_ACCESS_FROM_REQUEST"
    ]
  }'

Responses

OK

Bodyapplication/json

scheduledJobobject(ScheduledJob)

Represents a job that is scheduled for execution.

statusobject(Status)

Details about status or error of a service

Response

application/json

{
  "scheduledJob": {
    "id": "35f23c03-1c22-45fe-9484-3ffe769325de",
    "name": "Process vendor data.",
    "description": "I started this job to improve quality of our data.",
    "createdAt": "2020-08-31T16:47+00:00",
    "createdBy": "76248934691294444",
    "modifiedAt": "2020-08-31T16:47+00:00",
    "modifiedBy": "76248934691294444",
    "organizationId": "cdq_monitor",
    "workspaceId": "c074b9f3-abf0-4f8e-9a20-74deb6cfa2a4",
    "clientId": "12345678-1234-5678-1234-567812345678",
    "type": "DNB_JOB",
    "dockerImageUrl": "docker-image-url",
    "resourceRequest": { … },
    "environmentVariables": [ … ],
    "labels": [ … ],
    "volumes": [ … ],
    "locks": [ … ]
  },
  "status": {
    "code": "200",
    "technicalKey": "OK",
    "details": [ … ],
    "path": "/v2/businesspartners/lookup",
    "timestamp": "2019-08-31T16:47+00:00"
  }
}

List Scheduled Jobs

Request

List scheduled jobs that meet the querying criteria.

Query

startAfterstring(StartAfter)

Indicator for the next page.

Example: startAfter=5712566172571652

limitinteger(Limit)

Maximum number of scheduled jobs to return. Used for pagination.

Example: limit=100

featuresOnArray of strings(ScheduledJobsReadFeatureParam)

Enable additional features when fetching jobs. Supported features:

IGNORE_ORGANIZATION_FILTER - allows to read scheduled jobs from any organization

Items Value	Description
IGNORE_ORGANIZATION_FILTER	Ignore organization filter. Allows reading scheduled jobs from any organization.

Example: featuresOn=IGNORE_ORGANIZATION_FILTER

organizationIdstring(OrganizationId)

The organizationId parameter set in the scheduled job.

Example: organizationId=cdq_monitor

jobTypeArray of strings(JobTypeParam)<= 100 items

The kind of job described as JobType.

Items Enum Value	Description
AML_MAINTENANCE_JOB	AML maintenance job
BANK_DOCUMENT_DATA_INGESTION_JOB	Bank document data ingestion job
BANK_TABULAR_DATA_INGESTION_JOB	Bank tabular data ingestion job
BANK_TREE_DATA_INGESTION_JOB	Bank tree data ingestion job
BANKACCOUNT_DATA_CONFIRMATION_JOB	Bankaccount data confirmation job
BANKACCOUNT_DATA_IMPORT_JOB	Bankaccount data import job
BANKACCOUNT_DATA_STATISTICS_JOB	Bankaccount data statistics job
BANKACCOUNT_DATA_WHITELIST_CURATION_JOB	Bankaccount data whitelist curation job
BANKACCOUNT_DATA_WHITELIST_IMPORT_JOB	Bankaccount data whitelist import job
BATCH_LOOKUP_JOB	Batch lookup job

Example: jobType=BANKACCOUNT_DATA_CONFIRMATION_JOB

curl -i -X GET \
  https://idp.cdq.com/_mock/apis/jobs-management-api/api-v2/v2/scheduledjobs

Responses

OK

Bodyapplication/json

scheduledJobPageobject(ScheduledJobPage)

Page for displaying Scheduled Jobs.

statusobject(Status)

Details about status or error of a service

Response

application/json

{
  "scheduledJobPage": {
    "limit": "100",
    "total": "67",
    "startAfter": "5712566172571652",
    "nextStartAfter": "5712566172571652",
    "values": [ … ]
  },
  "status": {
    "code": "200",
    "technicalKey": "OK",
    "details": [ … ],
    "path": "/v2/businesspartners/lookup",
    "timestamp": "2019-08-31T16:47+00:00"
  }
}

Poll Scheduled Job

Request

Poll a scheduled job by its ID.

Path

jobIdstring(JobId)required

Unique identifier of the job.

Example: 35f23c03-1c22-45fe-9484-3ffe769325de

curl -i -X GET \
  https://idp.cdq.com/_mock/apis/jobs-management-api/api-v2/v2/scheduledjobs/35f23c03-1c22-45fe-9484-3ffe769325de

Responses

OK

Bodyapplication/json

scheduledJobobject(ScheduledJob)

Represents a job that is scheduled for execution.

statusobject(Status)

Details about status or error of a service

Response

application/json

{
  "scheduledJob": {
    "id": "35f23c03-1c22-45fe-9484-3ffe769325de",
    "name": "Process vendor data.",
    "description": "I started this job to improve quality of our data.",
    "createdAt": "2020-08-31T16:47+00:00",
    "createdBy": "76248934691294444",
    "modifiedAt": "2020-08-31T16:47+00:00",
    "modifiedBy": "76248934691294444",
    "organizationId": "cdq_monitor",
    "workspaceId": "c074b9f3-abf0-4f8e-9a20-74deb6cfa2a4",
    "clientId": "12345678-1234-5678-1234-567812345678",
    "type": "DNB_JOB",
    "dockerImageUrl": "docker-image-url",
    "resourceRequest": { … },
    "environmentVariables": [ … ],
    "labels": [ … ],
    "volumes": [ … ],
    "locks": [ … ]
  },
  "status": {
    "code": "200",
    "technicalKey": "OK",
    "details": [ … ],
    "path": "/v2/businesspartners/lookup",
    "timestamp": "2019-08-31T16:47+00:00"
  }
}

Update Scheduled Job

Request

Update specific fields in a scheduled job.

Path

jobIdstring(JobId)required

Unique identifier of the job.

Example: 35f23c03-1c22-45fe-9484-3ffe769325de

Bodyapplication/jsonrequired

scheduledJobobject(ScheduledJobUpdate)required

ScheduledJob update model.

scheduledJob.namestring(JobName)

Name of a Job.

Example: "Process vendor data."

scheduledJob.descriptionstring(JobDescription)

Detailed description of a Job.

Example: "I started this job to improve quality of our data."

scheduledJob.clientIdstring(ClientId)

Uniquely identifying ID of the client.

Example: "12345678-1234-5678-1234-567812345678"

scheduledJob.typestring(JobTypeParam)

The type of the job.

Enum Value	Description
AML_MAINTENANCE_JOB	AML maintenance job
BANK_DOCUMENT_DATA_INGESTION_JOB	Bank document data ingestion job
BANK_TABULAR_DATA_INGESTION_JOB	Bank tabular data ingestion job
BANK_TREE_DATA_INGESTION_JOB	Bank tree data ingestion job
BANKACCOUNT_DATA_CONFIRMATION_JOB	Bankaccount data confirmation job
BANKACCOUNT_DATA_IMPORT_JOB	Bankaccount data import job
BANKACCOUNT_DATA_STATISTICS_JOB	Bankaccount data statistics job
BANKACCOUNT_DATA_WHITELIST_CURATION_JOB	Bankaccount data whitelist curation job
BANKACCOUNT_DATA_WHITELIST_IMPORT_JOB	Bankaccount data whitelist import job
BATCH_LOOKUP_JOB	Batch lookup job

Example: "BANKACCOUNT_DATA_CONFIRMATION_JOB"

scheduledJob.dockerImageUrlstring(JobDockerImageUrl)non-empty

The URL of the Docker image.

Example: "docker-image-url"

scheduledJob.resourceRequestobject(ResourceRequest)

Resource request.

scheduledJob.environmentVariablesArray of objects(EnvironmentVariable)<= 100 items

Max 512 kB allowed.

scheduledJob.labelsArray of objects(Label)<= 50 items

List of labels.

scheduledJob.volumesArray of objects(Volume)<= 50 items

List of volumes.

scheduledJob.locksArray of objects(Locks)<= 5 items

Enables job locking. Prevents from running duplicated jobs of the same locks set.

Checks against NOT FINISHED, CANCELED nor FAILED jobs.

When tag=DATA_MONITOR is provided together with locks, then job is locked only against another DATA_MONITOR jobs.

Example: [ {"key": "jobType", "value": "DATA_CURATION_JOB"}, {"key": "storageId", "value": "87621345678976543245678965432"}, {"key": "dataSourceIds", "value": hash(["dataSourceId1", "dataSourceId2", "dataSourceId3", "dataSourceId4", "dataSourceId5", "dataSourceId6"])} ]

featuresOnArray of strings(ScheduledJobUpdateFeatureParam)

List of features to be activated.

Items Value	Description
SKIP_ACCESS_RESTRICTION	Update scheduled jobs from any organization regardless of access restrictions. Feature usage is only for dedicated users.

Example: ["SKIP_ACCESS_RESTRICTION"]

curl -i -X PUT \
  https://idp.cdq.com/_mock/apis/jobs-management-api/api-v2/v2/scheduledjobs/35f23c03-1c22-45fe-9484-3ffe769325de \
  -H 'Content-Type: application/json' \
  -d '{
    "scheduledJob": {
      "name": "Process vendor data.",
      "description": "I started this job to improve quality of our data.",
      "clientId": "12345678-1234-5678-1234-567812345678",
      "type": "BANKACCOUNT_DATA_CONFIRMATION_JOB",
      "dockerImageUrl": "docker-image-url",
      "resourceRequest": {
        "memoryLimit": 1024,
        "memoryRequest": 1024,
        "cpuLimit": 0.2,
        "cpuRequest": 0.1,
        "xmxLimit": 512
      },
      "environmentVariables": [
        {
          "key": "STORAGE_ID",
          "value": "123456789"
        }
      ],
      "labels": [
        {
          "key": "my label",
          "value": "my job"
        }
      ],
      "volumes": [
        {
          "mountPath": "/path",
          "subPath": "subpath"
        }
      ],
      "locks": [
        {
          "key": "jobType",
          "value": "SUBSCRIPTION_FAT_JOB"
        }
      ]
    },
    "featuresOn": [
      "SKIP_ACCESS_RESTRICTION"
    ]
  }'

Responses

OK

Bodyapplication/json

scheduledJobobject(ScheduledJob)

Represents a job that is scheduled for execution.

statusobject(Status)

Details about status or error of a service

Response

application/json

{
  "scheduledJob": {
    "id": "35f23c03-1c22-45fe-9484-3ffe769325de",
    "name": "Process vendor data.",
    "description": "I started this job to improve quality of our data.",
    "createdAt": "2020-08-31T16:47+00:00",
    "createdBy": "76248934691294444",
    "modifiedAt": "2020-08-31T16:47+00:00",
    "modifiedBy": "76248934691294444",
    "organizationId": "cdq_monitor",
    "workspaceId": "c074b9f3-abf0-4f8e-9a20-74deb6cfa2a4",
    "clientId": "12345678-1234-5678-1234-567812345678",
    "type": "DNB_JOB",
    "dockerImageUrl": "docker-image-url",
    "resourceRequest": { … },
    "environmentVariables": [ … ],
    "labels": [ … ],
    "volumes": [ … ],
    "locks": [ … ]
  },
  "status": {
    "code": "200",
    "technicalKey": "OK",
    "details": [ … ],
    "path": "/v2/businesspartners/lookup",
    "timestamp": "2019-08-31T16:47+00:00"
  }
}

Delete Scheduled Job

Request

Deletes a scheduled job by its ID.

Path

jobIdstring(JobId)required

Unique identifier of the job.

Example: 35f23c03-1c22-45fe-9484-3ffe769325de

Query

featuresOnArray of strings(ScheduledJobDeleteFeatureParam)

Features to be activated on during deleting Scheduled Jobs.

Items Value	Description
SKIP_ACCESS_RESTRICTION	Delete scheduled jobs from any organization regardless of access restrictions. Feature usage is only for dedicated users.

Example: featuresOn=SKIP_ACCESS_RESTRICTION

curl -i -X DELETE \
  https://idp.cdq.com/_mock/apis/jobs-management-api/api-v2/v2/scheduledjobs/35f23c03-1c22-45fe-9484-3ffe769325de

Responses

OK

Bodyapplication/json

statusobject(Status)

Details about status or error of a service

Response

application/json

{
  "status": {
    "code": "200",
    "technicalKey": "OK",
    "details": [ … ],
    "path": "/v2/businesspartners/lookup",
    "timestamp": "2019-08-31T16:47+00:00"
  }
}

Jobs Management API (2)

Jobs

Job Results

Processing logs

Scheduled Jobs

Create Scheduled Job

Request

Responses

Share the feedback!

List Scheduled Jobs

Request

Responses

Share the feedback!

Poll Scheduled Job

Request

Responses

Share the feedback!

Update Scheduled Job

Request

Responses

Share the feedback!

Delete Scheduled Job

Request

Responses

Share the feedback!

Jobs Management API (2)

Jobs

Job Results

Processing logs

Scheduled Jobs

Create Scheduled Job

RequestExpand all

Responses

Share the feedback!

List Scheduled Jobs

Request

Responses

Share the feedback!

Poll Scheduled Job

Request

Responses

Share the feedback!

Update Scheduled Job

RequestExpand all

Responses

Share the feedback!

Delete Scheduled Job

Request

Responses

Share the feedback!

Request

Request