Xero Stress Test
Testing was carried out to answer questions about how well Xero performs under higher loads. GrowthPath’s concern was for medium-sized wholesalers. Xero talks about limits as low as 1000 transactions a month. Testing showed Xero performs well at much higher transaction volumes.
After discussions with Xero and based on our own experience, GrowthPath believes that businesses can interpret the advisory “limit” of 1000 invoices a month as advisory, not a term of service. In January 2017, Xero published API limit changes that allow much higher loads than the advisory limits. Hence, stress testing is relevant because you will reach the practical limits of Xero before you exceed API limits.
Test Setup
Section titled “Test Setup”Testing was carried out in June 2015 and updated in November 2018 with much higher loads, in preparation for a talk at Melbourne AccountTech 2018.
- Standard Xero trial account, set to Australian locale
- Browser: Firefox on a fast Linux notebook
- Data created with Python scripts, automatically imported via CSV files
- Browser automation used for bank recs, document approvals, and simultaneous invoice entry (Python & Splinter)
- Some uploads via the API mainly to load Xero for testing
- Network: ranged from 100 Mb/s office connection to 5 Mb/s Indonesian hotel wifi
For a wholesaler scenario, Xero is the backend to a cloud inventory package. Xero’s inventory module was not used.
Test Results
Section titled “Test Results”Phase 1: Report Performance (Offline Use)
Section titled “Phase 1: Report Performance (Offline Use)”The testing database has data spread over 3.5 years, including >30K bank lines (mostly reconciled), >30K AP documents, and sales data spread over 50 customers.
Part 1
Section titled “Part 1”| Load Level | Low | Low | Low | Medium | High | Extreme |
|---|---|---|---|---|---|---|
| AR Invoices (@25 lines) | 250 | 500 | 1,000 | 6,000 | 11,000 | 16,000 |
| AR Lines | 6,250 | 12,500 | 25,000 | 150,000 | 275,000 | 400,000 |
| Task | Low | Low | Low | Medium | High | Extreme |
|---|---|---|---|---|---|---|
| Enter Sale | FAST | FAST | FAST | FAST | FAST | FAST |
| P&L, Month/Qtr | FAST | FAST | FAST | FAST | FAST | FAST |
| P&L, Sales drill down | FAST | BANDWIDTH | BANDWIDTH | BANDWIDTH | BANDWIDTH | BANDWIDTH |
| Simple BAS, Month | FAST | FAST | FAST | FAST | FAST | FAST |
| Simple BAS Audit | SLOW | SLOW | SLOW | TIMEOUT | ||
| Balance Sheet | FAST | FAST | FAST | FAST | FAST | FAST |
Part 2
Section titled “Part 2”| Load Level | Low | Medium | High |
|---|---|---|---|
| AR Invoices | 750 | 3,000 | 33,000 |
| AR Lines | 18,750 | 75,000 | 825,000 |
| Task | Low | Medium | High |
|---|---|---|---|
| Enter Sale | FAST | FAST | FAST |
| P&L, Month/Qtr | FAST | FAST | FAST |
| P&L, Sales drill down | BANDWIDTH | BANDWIDTH | TIMEOUT |
| Simple BAS, Month | FAST | FAST | FAST |
| Balance Sheet | FAST | FAST | FAST |
Result Key
Section titled “Result Key”| Rating | Definition |
|---|---|
| FAST | < 10 seconds |
| SLOW | < 60 seconds |
| VERY SLOW | > 60 seconds but succeeds |
| TIMEOUT | Timeout failure |
| BANDWIDTH | Performance limited by network speed, not Xero. Once download began, confirmed not a timeout |
Simultaneous Users
Section titled “Simultaneous Users”Xero assigns a next invoice number when a user starts entering an invoice. This number is not reserved, so if another user starts a new invoice, it will get the same number. With more than two users entering invoices at the same time, invoices can be overwritten. There is no reliable workaround apart from using a front-end system for order entry.
Conclusions
Section titled “Conclusions”- Total transaction load is not a factor for reporting performance — large historical data does not degrade performance
- Reporting performance gets slow only when reports download many rows, and the bottleneck is often bandwidth, not Xero
- The Simple BAS Audit report is the weakest link — it is the first report to timeout under extreme loads
- The practical upper limit is approximately 300K AR lines per month (about 60,000 five-line invoices) using the Simple BAS
- Invoice entry is always fast regardless of load
- Simultaneous browser-based order entry by 3+ users risks data loss due to invoice number conflicts
Workaround: Consolidation
Section titled “Workaround: Consolidation”The most common workaround is to consolidate entry via a separate system. A retailer uses a POS system; a wholesaler uses a cloud ERP (such as Cin7 Core or Unleashed) which sends only consolidated sales to Xero.