In my earlier two blogs I looked at the increasing opportunity for vertical database technology and why telco databases present unique challenges. In this final blog in the series, I want to talk about the database we’ve built at MATRIXX to sit at the heart of our platform and why we did it.
MATRIXX wasn’t founded with building a new database technology in the plan – our original goal was to engineer a next generation charging platform that could could cater to the explosion in digital services and rich mobile experiences that followed the huge success of the iPhone. But when you start digging into where the bottlenecks are within traditional rating and charging applications, the process quickly becomes a substantial excavation. Is the problem to do with application design and business logic execution? Are things held up by replication and clustering technologies? Well, yes, and we had to solve both of those issues as well, but our findings took us deep into the database persistence layer itself. It turned out that the online transaction processing principles that have underpinned relational and other transactional database designs for the last 50 years were now the root cause of the fundamental performance problems that we needed to solve. The culprit? Database locking.
As we discussed earlier in this series, relational databases have no assumptions about the applications that are built on them: they are general purpose, flexible, and invaluable in that jack-of-all-trades role. But relational databases deal with transactional compliance by locking the affected data until each transaction is committed. Setting up and removing these locks creates significant overhead for a telco network where hundreds of millions of transactions are completed each hour. Because these transactions are spread over a large customer base, the actual probability of two transactions colliding is quite low, so incurring all this overhead for every transaction fails the “engineering 80:20 rule” – make the common case efficient and let the uncommon case pay the price. It doesn’t matter if the database is disk-based or in-memory, SQL or NoSQL, relational or object-based; if the system is guaranteeing integrity by locking all related data for every transaction, then in this vertical, it will deliver unpredictable latency and fail to scale.
This was the realisation that started our technology journey towards a whole new paradigm for database transaction management. We are proud of our Parallel-MATRIXX database technology and the way it consistently delivers the following characteristics:
Low, predictable latency.
The objects in our in-memory database are available to the business logic application using direct memory access – there is no locking, no serialization/deserialization, no network hops and no complex transformations. When the business logic reads an object there’s a very small “consistency test” to ensure the object isn’t “half updated” and then memory speed access is granted to the data.
The objects in our in-memory database are updated by the business logic using our patented non-blocking, contention-free algorithm. In a nutshell, the business logic can perform updates to the database as part of a transaction without any of the overhead of traditional locking or blocking, but still maintaining strict consistency guarantees. This makes our database hundreds to thousands of times more efficient for managing write-intense workloads like online charging and keeps the performance of each transaction consistent independent of other load on the system.
Complex processing without impacting performance.
Complex policy and charging operations can access many objects in the database – the subscriber, the device, the wallet, many balances, the products owned, the pricing information on those products, the group hierarchy, the groups’ wallets and purchased products, etc. Our contention free database aligned with our approach to executing business logic means the system is remarkably immune to increasing complexity within the subscriber data model or pricing design.
In summary, we’re glad we did it the way we did, because we’ve created a unique database technology that underpins and distinguishes the entire MATRIXX value proposition. Time and again we outpace even the most carefully tuned competitors. The value of our platform comes from the subscriber experience and business innovation it enables, but the raw power of our database is what allows us to offer those advantages.