SAP Concur Expense API performance

[ne0404]

Hello,

Each day I'm downloading employee expense data for the previous day from the Expense API, but I'm finding the performance of my procedure is not what I'd expect so wanted to run the logic past somebody in case there is something obvious I'm doing wrong.

As an example, on one day recently there were 2,291 individual expense items to collect across 290 expense reports submitted and it took around 17.5 minutes to collect and land this data using our Databricks cluster, which seems slow to me.

The procedure is doing the following:

1. Connect to {baseUrl}/api/v3.0/expense/reports?user=ALL&modifiedDateAfter={fromDate}&modifiedDateBefore={toDate} to get each report updated yesterday and the relevant header data.
2. For each ReportId above, extract the UserName and then connect to: {baseUrl}/profile/identity/v4/Users?filter=userName eq {userName} to get the UserId for each report.
3. For each ReportId in Step 1, connect to {baseUrl}/expensereports/v4/users/{userId}/context/TRAVELER/reports/{reportId}/expenses to get each individual expense item on each report.
4. For each expense item above, connect to 
   {baseUrl}/expensereports/v4/users/{userId}/context/TRAVELER/reports/{reportId}/expenses/{expenseId} to get the Tax info and connect to:

   {baseUrl}/expensereports/v4/users/{userId}/context/TRAVELER/reports/{reportId}/expenses/{expenseId}/allocations to get the allocation info.

Any feedback appreciated.

Thanks,

Nick

[KevinD]

@ne0404 what is it you are doing with all this data once extracted? 

I have another suggestion that could help you, but wanted to see what you are doing with this data.

[ne0404]

Hi Kevin,

The expenses data will go into a fact table and then be consumed by our Finance Department via a Power-BI report / dashboard.  I'm not involved in creating the actual report / dashboard but I'm guessing they will be able to quickly answer questions like, which vendors do we spend most on, what time of year will we spend most money etc and also quickly be able to see any unusual spikes.  I guess they already have this but the fact table will do all the aggregations and calculations for them so they can quickly get to the answers.

Thanks,

Nick

[KevinD]

@ne0404 you should be able to get all that information by using Cognos reporting as well. 

[ne0404]

Okay, thanks Kevin. I'll take a look.