Introduction
Recovering from a delayed launch is possible; however recovering from production problems, such as a payment issue (bug), a broken KYC flow, or an update failing in production, is much harder. Founders and product leaders alike are faced with this overwhelming task of how to ship software quickly and remain competitive while maintaining user trust, maintaining compliance and maintaining operation stability.
This is where a solid QA and release strategy comes in.
Appricotsoft does not see test (QA) as a last minute cleanup/cleanup phase. Instead we consider QA an integral part of the delivery process; transparent, structure and is linked to release readiness. This mindset correlates to how we view working as a company. We build software that we are proud of; establish very high benchmarks for the way good software should be delivered; and execute the software delivery process to ensure that clients have clear understanding of the process makes the delivery as easy as possible for them. Our AI-first Unison Framework follows the same basic principle: AI can help execute but people own outcomes – therefore through providing weekly visibility the ability of a project to be a black box is minimised.
If you’re thinking about building a fintech product and would like a release model that is founder-friendly, realistic and scalable, then use the following guidelines to help guide you through your approach to testing, environments, release gates and go-live discipline.
Why QA in Fintech Is Different
Fintech is unique when it comes to quality assurance (QA) for a few reasons:
For any mobile application, QA is required; however, the ability to mitigate risk differs in developing financial mobile applications versus social or entertainment mobile applications. For example, for a social application, if there is a minor display issue for a day or two, that would be acceptable. However, for a financial application, if a user is confused regarding their balance, fees, transfers, identity verification, transaction status, etc., due to an error in the application, that could create call volume for support, failed transactions, lost customers, poor ratings, and/or regulatory issues.
Financial mobile application developers will ask many different types of questions beyond “is the feature working?” when conducting quality assurance, including:
- “Does the feature work in a reasonable manner across the many different real devices and operating system versions that our users will have?”
- “Does the feature behave in a reasonable manner to the end-user when a third-party provider of data is slow or will not respond?”
- “Will an error with the feature cause the end-user to suffer no adverse consequences?”
- “What explanation do the end-users receive in the event of an error with the feature?”
- “What processes/procedures are used to find the reason for a production defect?”
- “Is this version of the application build ready for production release?”
These questions determine how testing will occur for the application from the very beginning.
Realistic fintech timeline scenarios
A solid QA plan for developing a custom mobile application can generally be built upon three distinct layers of testing: unit tests, integration tests, and end-to-end tests. All three serve different purposes, and when development teams place too much emphasis on a single layer of testing, problems will typically ensue.
1. Unit Testing – A Swift Rebuttal of Business Logic
Unit tests are the first line of defense. They are used to validate discrete elements within the application: (i.e., calculations, formatting, validation rules, business logic, input checking).
The use of unit tests within a fintech application is particularly valuable in:
- Verifying calculations for transaction fees
- Verifying calculations for limits
- Verifying calculations for interest and/or balance logic
- Verifying permission logic
- Verifying input validation
- Verifying the mapping and transformation of third-party provider returns
- Verifying state transitions of payments, cards, or verification processes
Unit tests should be completed quickly and frequently. The main purpose of unit testing is not to ensure that the entire application is functional. The purpose of a robust set of unit tests is to identify regressions as early in the testing cycle as possible,e when the cost of fixing a line of code is cheapest.
As an example, if you change a rule affecting a transfer limit, a well-designed unit test suite would potentially identify unexpected impacts within the application long before that bug is discovered in the staging environment.
2. Integrated System Tests – Confirmation of Collaborative Behaviour of All Systems
Many fintech products will come up short when subjected to rigorous exams.
Integration Tests focus on determining how separate components of the application will work together through several respects, such as the call to back-end services, payment gateways, KYC-AML service providers, Notification service providers, Card Processor service providers, Bank Linking Partners, Analytics, and other /or internal API’s.
This is critical because the majority of fintech products are designed as orchestration products and not as isolated systems; generally, even a simple wallet or a Payments app depends on a variety of third-party services; the quality of the application experience from these services will depend on the degree to which these integrations were developed through integration testing.
Integration Tests should test for:
- successes and failures when initiating payments
- webhook processing
- handling of duplicate Events and idempotent actions
- responses from KYC decisions
- results of account linking
- timeout/retry processes
- partial provider failures
- reconciliation stateful updates
This is also why Fintech Industry is so dependent on Sandbox testing; the documentation provided by Stripe has extensive references urging the development/testing of integrations within a Sandbox environmentbeforeo going live, including payment flows, webhooks and simulated transactions; Plaid also provides a fully functional Sandbox for testing of account linking and API functionality without involving the use of any real or live financial data when first being tested.
As such, an effective Quality Assurance and Test plan extends beyond “happy path functionality” in the Sandbox to explicitly exercise declines, session expirations, interrupted onboarding, and poor data rates.
3. E2E Testing – Reviewing User Experience
E2E testing can determine if a product works by means of the way users are going to go through all steps within the system, including, but not limited to, hardware, user interface, back-end system, 3rd party integrations, notifications, and current state changes.
For example, in the financial technology sector, E2E tests usually focus on the workflows that are the most trust & revenue sensitive, for example:
- User registration & login
- KYC + ID verification
- Adding a card or linking to a bank account
- Sending money
- Receiving Money
- Payment confirmations
- Transaction History updates
- Password resets or step-up verification
- Support escalation flows
During the founder’s journey, many new businesses figure out the difference between a product that is ‘Technically Complete’ and a product that is “Release Ready”.
Just because a feature can pass developer testing doesn’t mean it will ever work correctly in the real world. Many variables in real life could break a feature. Examples include but are not limited to: UI (User Interface) issues that are specific to certain devices, “dead ends” that users can hit when trying to register or log in, >overlapping fields displayed by a keyboard, race conditions (this is when a user pushes a button before it is actually enabled), or ambiguous and unclear error statements.
This is why all E2E testing should be reserved for confirming the most important user journeys at the time of release, as these features do not work and could possibly impair the success of the launch.
The Importance of a Comprehensive Device Testing Strategy
A major pitfall when providing a mobile app development service is to underestimate the number of devices used as part of the mobile app device matrix.
Testing your app on only the latest iPhone and one recent Android device isn’t a realistic test strategy; in fact, it’s wishful thinking.
A practical device matrix for the development of fintech apps would take into account:
- iOS and Android platforms
- Current and previous versions of the operating systems
- Low, mid, and very high cost devices
- Different screen dimensions/sizes
- Biometric and non-biometric devices
- Different mobile network environments
- The different upgrade paths of the app from previous versions
The mobile devices you choose to test on are not necessarily exhaustive, but they need to represent a unique distribution of devices that resemble the actual users of your developed app and/or services and risk profile.
To reach the greatest coverage possible on Android devices, there are ways to do this effectively using cloud-based solutions for real device testing (e.g. testing on devices that are in production environments) through the use of real devices across a variety of Google-owned data-centers (e.g. Firebase Test Lab) using a configurable/expandable testing device matrix for broader validation and faster test delivery.
At Appricotsoft, we like to follow a layered approach with our device matrix; we establish a small but representative set (the “always test”) of mobile devices for each app release followed by expanding this testing device matrix based on higher risk change items, such as onboarding, biometrics, payments, and SDK upgrades.
Sandbox Testing: Helpful, But Not Always Reliable
When founders hear, “We tested it in the sandbox,” they often assume that the feature is “good to go.”
Not true.
Sandboxes are invaluable during development because they allow teams to verify that their integrations work and provide your team with the confidence that their work is safe (in terms of not using any actual money and not using live customer data).
For example, Stripe has test card numbers available for validating charges; it offers simulated transactions, allows dispute resolution (i.e., getting refunds), tests authentication scenarios, and validates webhooks through its testing environment. So does Plaid with its sandbox, which allows developers to test how their applications connect to financial institutions globally.
However…
There are limitations to testing in a sandbox environment.
Sandbox environments do not necessarily replicate the timing of production, edge case data scenarios, odd behavior of third-party providers, operational latency, or any other environmental constraints you might encounter in production. Because of this, mature release strategies will generally use sandbox testing only for validation and will augment their sandbox verification strategy through the following:
- Testing-related mocked failure scenarios
- Testing in staging environment
- Smoke testing in production
- Gradual release strategy
- Monitoring after the software is released (post-release)
The combination of the above is much stronger than just relying on a sandbox to validate all the features and functionality of your application.
Automating Regression Testing: Safeguarding Existing Functionalities
As a Fintech app continues to scale, relying solely on human testing becomes uneconomical and too brittle.
Regression testing automation allows your organization’s team members to operate confidently by safeguarding the flows required to continue operating through multiple production releases while accommodating new features introduced into the application.
A sensible regression suite generally contains:
- user login processes and authentication
- onboarding procedures for new users
- all core methods for making payments and/or transferring money
- transaction list view and transaction detail view
- payment limitations and confirmation messages
- support or contact us pages
- push notification trigger statements
- Administrative or operational actions based on Roles
The intent is NOT to automate everything,g but instead primarily automate the most business-critical and repeatable series of tests first.
This is especially relevant to organizations that are developing rapidly while also supporting multiple platforms, developing mobile applications utilizing React Native or other technologies, where any single change will impact multiple devices’ functionality at once. Automated testing provides teams with confidence in producing quality software during each release cycle.
Automated software testing provides teams with visibility into their application quality throughout the application delivery process, as demonstrated by Appricotsoft. We utilize an Unison Framework to create reusable artifacts, definitive lists for completing project deliverables, documented risk associated with delivery, and ensure that quality is “built into” during our workflow, versus “pushed out” until all development and testing activities are completed.
Release Gates: Requirements Before You Release
In order for the next build to proceed, a release gate requires a checklist of specific criteria that need to be fulfilled.
The absence of release gates often results in shipping decisions made by pressure, hope, or fatigue; whereas a defined release gate makes such decisions much more evident.
An example of a fintech release gate could be:
Product Gate
- The scope of the product is locked.
- Acceptance criteria have all been met.
- Critical edge cases have been reviewed.
- There are no undefined ambiguities in critical paths.
Engineers Gate
- Code review has been conducted.
- All tests have passed.
- All migrations and feature flags have been validated.
- All observability or logging has been implemented as required.
Quality Assurance Gate
- All manual checks for the critical path have been conducted.
- All regression tests have been executed.
- All devices in the device matrix have been accounted for.
Security and Compliance Gate
- All secrets and configuration settings have been checked.
- No test data has leaked.
- The team has performed a review of all permissions.
- All high-risk functional changes have been assessed against the security baseline.
OWASP’s Mobile Application Security Verification Standard (MASVS) is the “gold standard” for wholesale validation of the mobile application security controls before release to live customers, allowing teams to systematically assess the security controls in their Mobile applications during testing, and before releasing them into production.
Operation Gate
- Ensure that the support teams are notified.
- Roll-back plan is prepared.
- Analytics and alerts are configured.
- The release owner has been assigned the responsibility.
- Post-release checks have been scheduled.
This is not about bureaucracy; this is good business to mitigate risk.
Staged Rollout Beats Big-Bang Release
Instead of doing all at once in the Fintech rollout strategy, being strategic is better. Platforms like Google Play and Aapple provide a method for getting software released via staged rollouts (also known as phased release). The staged rollout process takes into account that while everything may pass all tests and look fine prior to production, actual production usage may cause errors that did not occur during testing. A staged rollout allows for monitoring of items such as:
1. Error Rate
2. Payment Success Rate
3. Login Failure Rate
4. Support Request Rate
5. App Review Rate
6. User Dropoff Rate
Founders can use staged rollouts as a way to lower the risk of their app launch without drastically extending the delivery time.
QA Checklist for Fintech Software Releases
As a checklist to use before endorsing a release (the areas checked may change based on your business processes, but generally, you should review these).
Core Functionality
- Is the registration and login/logout/password reset working as expected?
- Is the KYC/onboarding flow functioning as anticipated?
- Are the payment/transfer/top-up of wallet flows being processed end-to-end?
- Are the transaction statuses being updated correctly?
- Are the confirmations/receipts/notices being presented correctly?
Error Handling
- Are the messages displayed clearly for failed payments?
- Is the retry logic working without any issues?
- Do timeouts and outages of providers fail gracefully?
- Will duplicate actions not generate multiple financial events?
Device and Platform Coverage
- Has the release been tested on the agreed-upon iOS devices and versions?
- Has the release been tested on the agreed-upon Android devices and versions?
- Are the layouts, keyboards, and prompts behaving as expected?
- Has the upgrade path been tested with the previous version?
Integration Validation
- Have the sandbox checks been done for all of the providers involved?
- Were the webhooks/callbacks/async events verified?
- Were the test credentials/test configurations removed from production?
- Have the analytics and monitoring events been verified?
Release Readiness
- Did the regression automation pass?
- Are there any open bugs? If so, do you know what severity level to place them?
- Is there a rollback plan documented?
- Has the support team been briefed on the scope of the release and known issues?
- Is there a staged rollout/phased release plan in place?
This is a lightweight checklist that is intentionally made that way – it aims to create clarity, not stacks of paperwork.
What Founders Should Be Ready For When Working With A Partner to Build a Delivery Solution
When selecting a mobile application development firm to manage a financial technology app, be direct and specific with questions concerning the quality assurance and the process for releasing code.
It’s unacceptable to receive a generalized response such as, “We do test everything”.
Instead, you might want to know the following:
- What type(s) of tests (unit testing, integration testing, end-to-end testing) do you perform?
- Which tests are automated, and which ones are manual?
- How do you come up with the table of devices you test on?
- How do you test whether or not a user can link their payment method to your application?
- At what point do you pass a release to your client?
- What will you do if a release needs to be rolled back?
- What is your process for releasing code to your clients and for monitoring the effectiveness of such releases?
A quality partner will be able to articulate this information in clear terms without utilizing technical jargon.
Culture is also important here; Appricotsoft has defined its internal values to be honest, responsible, and accountable, and strives to provide a quality product, as opposed to working around the system. So, to their clients, this generally equates to fewer times being surprised.
How Appricotsoft Monitors Quality Assurance and Release Management Strategies for Fintech Applications.
Appricotsoft views fintech app development as a trusted product rather than just a feature of the app.
Therefore, QA is built into the delivery from the beginning. Weekly demos are used to provide visibility into the progress of the project as well as identify risks early on, quality metrics will be linked to day-to-day implementation, and AI will help to assist in accelerating repetitive tasks such as writing test cases and ensuring consistency among testers; however, ultimately it is the responsibility of the team to make the final judgment on whether or not they have met all requirements for verification and release readiness.
In our typical practice we will implement:
- Measurable acceptance criteria before starting the build process.
- Perform testing at all levels of the application rather than solely at the UI level.
- Try to cover all supported device types based on risk for the product.
- Validate sandbox environments for any third-party integrations.
- Create regression tests for all critical workflows in the application.
- Maintain release verification checklists and complete a go/no-go on all releases prior to production implementation.
- Plan for a staged rollout rather than a push-and-pray.
We have also made a concerted effort to ensure our process can be easily understood by both non-technical founders and executive teams, you should be able to determine where a project is with regard to quality, areas of risk and the state of preparation for shipment.
This has been particularly important to Fintech companies, because although timelines are critical; however it is building trust over time that ultimately adds value to the business.
If you want a broader view of how to structure a product from the beginning, these Appricotsoft posts are a good next read: Fintech App Development Roadmap: From MVP to Scale and Fintech App Development Features Checklist: What to Build for MVP vs V2.
For external benchmarks, OWASP MASVS is a strong security baseline for mobile apps, and the official Google Play and Apple documentation on staged releases is worth bookmarking for release planning.
Conclusion
The point of a proper fintech quality assurance plan is not to perform more testing per se. It is testing the right things, in the right conditions, with the right release management practices.
And the combination that works best looks like this:
- business-logic unit tests for quick validation
- integration tests for provider and system behavior
- E2E tests for important user scenarios
- a realistic device matrix
- testing in sandbox environments and prudent release to production
- automated regression testing for key flows
- clear release gates
- staged deployment and post-release monitoring
This way, you will be able to go fast without breaking anything.
In case you need a fintech software development company that will approach development and testing the right way, Appricotsoft has the product design philosophy, technical expertise, and pragmatic quality assurance processes that you won’t have to struggle to understand. We care about building high-quality software, and delivering predictably.
If you are developing a fintech app and seeking a reasonable delivery roadmap and strategy, send us a software development estimate request, and we will discuss the most reliable ways from MVP to production release.
As per instructions, reply written within the specified format.