Applications that handle large amounts of data are naturally more prone to performance issues. The result can be slower operations, which in turn may have a huge negative impact on user experience. In our experience at iRonin, it’s best to invest in upgrading your app and architecture sooner rather than later, to avoid hurting your brand and to optimize your costs. Let’s dive into why we recommend this approach.
Big Data performance issues
The simple truth is that, despite the advances of modern technology, operations carried out on large amounts of data take time. And while speeding up data processing is a complex problem (which tends to result in micro-optimization), the root of it isn’t always just the fact that a lot of data is needed. Sometimes, the trouble begins much earlier - with the creation of unscalable software.
To combat this, it’s good to stay up to date regarding major software updates that your project may need. We recently covered the topic of limiting the risks that come with upgrading applications. Now, let’s take a look at what we can do for Big Data projects, especially in a highly regulated sector such as fintech.
Maintaining scalable applications
At iRonin, we use a number of strategies to make sure our clients’ application will scale well and be prepared to handle large volumes of data.
Separating migrations from code changes
Doing so leaves us the option to easily revert code changes at any time without worrying about locks. Migrations in Ruby on Rails, when they are being applied to a database table that already exists, lock the reads (in most cases) and sometimes even the writes (updates) to the given table. When no database migration is run, then there is no lock on the database.
Strong migration gems
By using the strong migrations gem, we can make sure we will write migrations effectively. The strong migrations gem catches unsafe migration code at the time of development, which lowers the risk of any kind of issue when the code is deployed to production.
Writing migrations effectively means doing so in a way that not only avoids problems, but also doesn’t cause reads or writes blocks. This is important when working with a lot of data, because database migrations tend to take more time.
Scheduled background processes
We recommend moving certain actions do the background and scheduling them using the push_bulk feature from sidekiq.
Using the database for calculations
Better performance can be achieved by completing amount calculations on the database.
Reducing the amount of data
This may sound simplistic, but often applications don’t use all the data available to them, or at least not for every operation. In such cases, we can apply a scope for database queries. It means that, when when the app performs payment calculation, for example, the scope is narrowed only to a given customer.
Code reviews
Reviewing code and making sure that the correct indexes are in place should be obligatory. Also, from time to time, it’s good to check your database to make sure that the indexes which were used previously are still being used. Sometimes, a certain functionality is modified or removed, but the indexes might be overlooked and left in the database. Invalid database indexes can cause a lot of unnecessary operations, which can lower the database’s performance.
Conclusions
As you can see, it’s often worth it to optimize your app’s code while you don’t have to handle that much data. Your database will grow, and if you don’t prepare for this, the app will stop working smoothly, resulting in poor customer experience. Much like with your physical health, the health of your app can be all about prevention.
Ready to consider upgrading your app to improve it’s performance? We can help.
iRonin’s experts have supported many clients through software updates and we continue to maintain projects with large user bases.
