Hello,
Each day I'm downloading employee expense data for the previous day from the Expense API, but I'm finding the performance of my procedure is not what I'd expect so wanted to run the logic past somebody in case there is something obvious I'm doing wrong.
As an example, on one day recently there were 2,291 individual expense items to collect across 290 expense reports submitted and it took around 17.5 minutes to collect and land this data using our Databricks cluster, which seems slow to me.
The procedure is doing the following:
- Connect to {baseUrl}/api/v3.0/expense/reports?user=ALL&modifiedDateAfter={fromDate}&modifiedDateBefore={toDate} to get each report updated yesterday and the relevant header data.
- For each ReportId above, extract the UserName and then connect to: {baseUrl}/profile/identity/v4/Users?filter=userName eq {userName} to get the UserId for each report.
- For each ReportId in Step 1, connect to {baseUrl}/expensereports/v4/users/{userId}/context/TRAVELER/reports/{reportId}/expenses to get each individual expense item on each report.
- For each expense item above, connect to
{baseUrl}/expensereports/v4/users/{userId}/context/TRAVELER/reports/{reportId}/expenses/{expenseId} to get the Tax info and connect to:
{baseUrl}/expensereports/v4/users/{userId}/context/TRAVELER/reports/{reportId}/expenses/{expenseId}/allocations to get the allocation info.
Any feedback appreciated.
Thanks,
Nick
