This is a guest post by Nehme Bilal from, Backend Engineer at EarnIn.
As an innovative technology organization in earned wage access, Earnin.com‘s mission is to build products for a more equitable financial system and a better life. Built for the unique needs of those living paycheck to paycheck, Earnin is designed to free people from the traditional bi-weekly or monthly payment cycle and help them take control of their money, starting from when they earn it.
In the United States, wages are paid out to the employees on bi-weekly or monthly basis. Which means that there is a lag between the time the work is performed and the time when compensation is paid out. With this liquidity gap, Americans accumulate over $50 billion in late and overdraft fees per year and often turn to alternative financial products like high-interest payday or pawn loans and cash advances. We provide a consumer-friendly app to help Americans avoid these products altogether.
We provide our community members (our users are referred as community members) with a modern and fully featured mobile application to aid them in their financial journey. Our users leverage our app to get quick access to their wages, keep an eye on their account balances, and manage their transactions. In this post, we share how we used Amazon Quantum Ledger Database (Amazon QLDB) to build a new ledger service to support these features.
The Microservices Challenge
Our mobile app is backed by microservices hosted on AWS. These microservices each own a private database (or no database at all for stateless microservices). Microservice implementation leads to a highly resilient, loosely-coupled, fault-tolerant backbone that supports our user-facing app. Although the microservices architecture has a lot of benefits, it makes it difficult for us to create an aggregated view of a user’s financial transactions due to the distributed nature of the databases. In fact, in order to aggregate financial transactions for a community member, we need to make API calls to many different microservices and merge the returned data. This ends up being an expensive and complex operation.
Since the data is distributed across many different databases, each using different technologies and design approaches, the data scientists on our machine learning teams face challenges when running system-wide queries. Additionally, we want to build a data warehouse without having to build cumbersome extract, transform, and load (ETL) processes or manage database instance storage.
The Earnin Ledger System
To overcome these difficulties, we decided to create an aggregated view of a community member’s financials that we call the ledger. Every microservice that runs business logic affecting a user’s financials is also responsible for updating the ledger in an eventually consistent manner. In addition to being useful for data analytics, the ledger is also used to serve user-facing APIs and generate financial statements.
The following diagram illustrates the architecture of our updated ledger service using Amazon QLDB.
Earnin’s QLDB implementation
Community member requests generated through the app are served by microservices that write to their local database as well as to Amazon QLDB. Every change to the QLDB ledger is reliably propagated to Amazon Aurora MySQL-Compatible Edition through Amazon Kinesis Data Streams, leveraging the built-in support of change streams in QLDB. The Aurora database is used to serve application read-only APIs where we need advanced queries that only SQL can offer and can tolerate eventual consistency. Furthermore, the database is replicated to Amazon Redshift which is used by our data science team for business intelligence purposes.
In accordance with the classic accounting practice, we have built our ledger based on double entry accounting ledgers. Every ledger-transaction is a sequence of debits and credits that are balanced, and are stored in a single QLDB entity. In other words, the sum of credits in a given transaction must be equal to the sum of debits in that same transaction. The debits and credits in a given transaction are applied to various accounts. We have created the ledger accounts under categories such as asset, liability, expense, and income. In addition to being a transaction log, we also use the ledger service to store balances. In fact, the balances of all accounts for a given community member are stored in a single QLDB entity, which is updated every time a new transaction is executed on the ledger for the community member.
Ledger Requirements and Capabilities
To implement the Earnin ledger service, we needed a data management service with the following capabilities:
Immutable and verifiable transaction log – Given that the ledger is used to serve user-facing APIs and to generate monthly statements, it’s important for us to keep an immutable and verifiable journal that reflects every update to the ledger. The updates are recorded as actions performed and are generally referred to as a journal entry. Once recorded, the journal entry should not be modified; and any further change to the account should be a new journal entry. Journal entries act as a trail of actions performed and yield the current view of the ledger. Ideally, we do not want the journal entries to be accessible without special permissions. More importantly, under no circumstances should anyone be able tamper with/modify these journal entries.
Automatically scalable datastore – Because every financial transaction is written to the ledger, we want the datastore to scale automatically. We do not want to worry about configuring DB server, provisioning capacity or configuring read/write capacity units.
The ability to modify multiple entities in a single ACID transaction – When we add a new transaction to the ledger, it affects the balance of several accounts. Thus, every transaction results in a change to the balances stored in the ledger database. To make sure that the journal transactions and balances are strongly consistent, we want to update all the related entities in a single ACID transaction.
The ability to retrieve previous balance for an entity based on a time range – To generate monthly statements (and for other accounting purposes), we need the ability to fetch the balances of a given community member at a specific point in time. Given that these balances are stored in QLDB (the balance entity), we can simply retrieve previous balance entities to get the balances at a given point in time.
Optimistic Concurrency and serialized operations– We often query the ledger balance before making business decisions, which results in a new ledger transaction. For example, if a user wants to transfer some funds from their Earnin account to an external account, we first check their balances to make sure they have sufficient funds and then create a new ledger transaction reflecting the transfer of funds. This can lead to a race condition if another transaction occurs after we read the account balance but before we record the new transaction. To avoid this issue, we require optimistic concurrency control where the database does not acquire locks on the account balance, and operate in full serializable operation.
Ability to subscribe to the ledger changes – We have several microservices that would be interested in reacting to account balance changes. Ensuring that a notification is published to a publish/subscribe system such as Amazon Simple Notification Service (Amazon SNS) or Amazon Kinesis after an account balance is published is error-prone. Ideally, we want the storage system to reliably handle this out of the box. In addition, the ability to subscribe to a change feed allows us to replicate the main database to a secondary storage if needed.
Among the many managed databases available out there, we mainly considered MySQL, Amazon DynamoDB, and Amazon QLDB. The following table shows a side-by-side comparison of these services with respect to the feature set we’re looking for.
Feature
RDS/Aurora MySQL
Amazon DynamoDB
Amazon QLDB
Immutable and verifiable transaction log
No
No
Yes
Serverless Database with automatic scaling
No
Yes
Yes
Modify multiple entities in an ACID transaction
Yes
Yes
Yes
Retrieving an older revision of an entity based on a time range
No
Yes*
Yes
Ability to subscribe to a change feed
No**
Yes
Yes
Support for optimistic concurrency
No
Yes
Yes
*Yes – through Amazon DynamoDB Point In Time Recovery
**No –AWS Database Migration Service facilitates change data capture.
Based on this comparison, Amazon QLDB is the obvious choice because it supports the features that we need for the ledger service. We indeed implemented our ledger service using QLDB and were able to meet our objectives discussed above.
Conclusion
In this post, we shared how Earnin updated our ledger system with Amazon QLDB to meet the needs of our mobile app, data science, and accounting teams. Using QLDB we were able to create an aggregated, comprehensive, immutable, and cryptographically verifiable view of our community members’ financial transactions. This means that any change to our community members’ financial transactions is reliably tracked, and accessed through QLDB’s change history.
Having all the transactions stored in one place makes it more efficient to serve our mobile application APIs. In addition, our data-scientists were delighted to rely on one database instead of many. We also took advantage of QLDB’s ability to retrieve an older version of an entity to generate monthly statements and for accounting purposes.
Furthermore, the serverless nature of QLDB and its ability to scale automatically meant that we never have to worry about reaching a limit on the number of users we can support while keeping our application fast and responsive.
When it comes to querying data, QLDB is not as powerful as relational databases and supports only a subset of SQL queries. That said, thanks to its built-in support of change streams, we were able to reliably replicate every QLDB transaction to Aurora MySQL, which we used to serve APIs that require complex queries.
Overall, adopting QLDB made Earnin’s mobile application more reliable, secure and fast. It also made our backend simpler and allowed our data scientists to be more efficient. As of September 2022, Earnin has performed more than 125 million transactions and provided access to $10 billion in earnings for its members.
About the authors
Nehme Bilal is a Staff Backend Engineer at EarnIn (previously worked at Amazon and Microsoft) with nearly 15 years of experience. Nehme leads the development of several backend services at EarnIn. He is also passionate about establishing processes and guidelines to make his team more efficient at building high quality software.
Shilpa Bondale is a seasoned technology professional with 20 years of experience in Silicon Valley, currently serves as a Solutions Architect at Amazon Web Services. In addition to her professional achievements, she is an active member and advisor for Women Who Code, further demonstrating her commitment to advancing diversity and inclusivity in the tech industry.
Read MoreAWS Database Blog
