Uncovering the Differences Between Open Banking and Screen Scraping

Exploring the future of regulated and unregulated digital data sharing methods in Australia


Download the Report

 

Technology has made its impact on history

It has shaped the way we perceive the world through the lenses of convenience. We have fashioned ourselves tools to better manipulate our surroundings for practicality and ease, and such tools have undergone modifications to improve our experiences. Data sharing has now evolved from sharing paper statements and screen scraping to consumer-friendly Open Banking. The benefits of Open Banking as a data sharing method go beyond efficiencies associated with digital transfer of information. Indeed, this new regulatory framework enhances the consumer’s security, confidence and control over their data. At the same time it creates both opportunities and challenges for companies and it appears inevitable that the industry will gradually adapt by fading out screen scraping data sharing methods.

In this analysis, Adatree explores the evolution and future of data sharing with an outline of the key differences between screen scraping and Open Banking. 


The history of digital data sharing

Since the 1980s, financial institutions such as banks and credit unions have commonly used screen scraping to extract data from legacy systems later to be adapted into modern applications. This technique was an expedited digital process as opposed to manually sharing, entering, and analysing data; additionally, companies were able to gather information from said data and cater to their customers accordingly. Digital data sharing has now evolved to Open Banking, or the Consumer Data Right in Australia. This is a regulator driven approach to mandating types of data with specific SLAs and common technical standards for third parties to participate and access data. 

Adatree CDR Comparison Table (1)
what is Screen Scraping?

To obtain a variety of services, customers provide their bank login credentials, such as their username and passwords, to a company—often times a bank or a lender—that facilitates the scraping of a customer’s information. These third parties then log into web-based platforms as though they were the customers themselves, seeing what their customers would see in the online banking “screen”. Once logged in, the third party would “scrape” the data. 

The “scraping” extracts the data from a digital display, collects it as raw text, and converts it for use in other applications. Data can continuously be “scraped” and updated from the customer’s accounts when refreshed. To facilitate this automatic process, screen scraping utilises image processing, so the accuracy may not be perfect.

For years, companies used screen scraping to extract data from legacy systems of financial institutions; they then transfer that data to modern applications to be used for other purposes. For example, a budgeting app can import a customer’s ongoing transactions, often with a delay.

Screen scraping provides not only for improved efficiency in obtaining and storing data, it is also fairly easy to implement at any third party institution seeking data. However, these benefits come at a disadvantage to consumers in terms of transparency and control.


Advantages of Screen Scraping Sit Largely With The Business

Screen scraping may be considered superannuated, especially since it doesn't have the same transparency and immediacy as Open Banking does, as outlined below. However, unlike in the UK where screen scraping was first banned, it is still legal in Australia. In fact, according to the Australian Securities and Investments Commission (ASIC), screen scraping is a reliable and safe method, hence unnecessary to be proscribed. Yet despite the attested “reliability” and “safety”, is screen scraping truly a method that should be perpetuated? 

Screen scraping still does have many advantages. The setup process is fast for the third party because it bypasses the data holder’s systems and data sharing permissions. It is straightforward as it only requires customers to share their login credentials. The customer data can then be stored digitally and accessed by the third party at any frequency and without any restrictions. Finally, there is no expiration date for accessing the data.

These benefits largely stem from the lack of barriers, regulation or standards for security, data storage, data usage, or ongoing access. Screen scraping is currently more popular for accessing bank transactions than Open Banking. Yet with the regulatory shifts and the associated shifts in customer preferences, this is likely to change.


Customer Disadvantaged by Screen Scraping Methods

Screen scraping has been widely adopted since it has long been one of the few viable digital sharing methods. Whilst it carries a range of benefits to third parties, the current shift towards Open Banking reflects the disadvantages of this method to customers.

Screen scraping requires the customer relinquish their control over their own data once the login details are shared. This results in a lack of transparency and requires a high level of trust for the third party on behalf of the customer. The customer does not know how and where the data will be stored; whether the data will be sold or shared, with whom, and for what purpose, everything remains unknown. If the customer decides for their data to be deleted, they lack legal recourse. The only way that customers can prevent this continued data sharing is to change their password.

It is not surprising that a significant number of consumers do not fully grasp the extent to which their information is being accessed, shared, stored and retained—or by whom—when they use financial apps. -PNC Bank

Even if a level of trust is established between the customer and the third party, and that should be based upon the company’s commitment to security, screen scraping is not infallible when it comes to breaches. Sharing credentials with a third party also is in violation of the terms and conditions of many institutions, which may increase the liability of the consumer. 

While screen scraping uses HTTP encryption over traffic, it is ultimately an unregulated practice with technical standards and customer controls falling significantly short of the requirements for third parties receiving data through Open Banking. 


Open Banking as a Global Trend

The Payment Services Directive 2 (PSD2) is a regulation that enforces Open Banking in Europe since January 13, 2018. The intent behind the directive is to push for competition, innovation, and quality in the financial industry, which in turn would lead to a more secure financial data sharing system. With this, data sharing would undergo a trailblazing reformation.

While Europe may have had a headstart when it comes to Open Banking, countries across the globe have been finalising similar preliminaries of their own standards while appraising PSD2’s impact on the continent. To the east, countries like China, India, and Singapore have adopted a data protection and exchange ecosystem. In the US, despite there being no Open Banking government mandate, companies are launching market-driven initiatives; meanwhile, Canada is entering the latest stages of establishing an Open Banking regime. 

As for Australia, the Department of Treasury has gone underway to introduce a similar regulatory framework as well. On February 13, 2019, the Australian government instituted new data sharing legislation: the Consumer Data Right (CDR). According to the Australian government’s Review into Open Banking, the CDR, often referred to as Open Banking, is defined as the right of consumers to have open access to their data, which is implemented in banking, energy, and telecommunications sectors.

This introduces a revolutionised framework of how industries can access and leverage consumers' financial transactions, granting consumers new controls over whom they wish to provide their information to, what types of data they wish to disclose, the purpose, what can be done with the data, and more. 


How Open Banking Works

The Consumer Data Right is a regulated ecosystem of data sharing. For the purpose of this analysis, the CDR will be referred to as Open Banking. Organisations that hold consumer data are referred to as Data Holders, and as the industries for Data Holders are designated, they need to comply with data sharing obligations. They need to allow consumers to share their data with accredited third parties, which are referred to as Accredited Data Recipients (ADRs). 

There are rules and standards for what customers and data recipients can view and do in terms of data sharing as well as the underlying technical standards. This ensures that consumers know what data is shared, with whom, for how long, and for what purpose. This is managed through an interface called a consent dashboard, which the customer can operate through both the Data Holder and the ADR. 

Consumers have a consistent consent process when sharing data through Open Banking

Consumers have a consistent consent process when sharing data through Open Banking

Any organisation aspiring to be an ADR must become accredited by meeting specific technical, business, security standards and rules, and customer experience guidelines. Any organisation that meets the accreditation criteria can be an ADR regardless of industry. 

Open banking is built on application programming interfaces (APIs), which provide a programmatic interface where applications can interact with other applications. Open Banking utilises secure APIs provided by the Data Holder organisations so Data Recipients can access the data consistently, reliably, and in near real-time.  Unlike screen scraping, Open Banking can access up to seven years of data, depending on the organisation. Ultimately, any organisation with more complex financial modelling and analysis needs will find that Open Banking is more suitable for these purposes than screen scraping.


consumer & technical advantages with Open Banking

Aside from technical improvements in speed of obtaining data (near real-time), Open Banking addresses many of the customer-side issues in screen scraping and empowers the customer with new rights and capabilities. 

Industry Sandbox login

In terms of control, the customer now benefits from a clear and consistent consent process when initiating and managing the sharing of the data. There is no need to share passwords during the consent process. The ability to manage, withdraw, and expire consent provides for flexibility and the right to deletion restores ownership of the data from the party to the customer. The customer can also designate the duration for access to their data.

Transparency is also enhanced as the regulation requires that the customer is in full knowledge of any party that receives the data, whether as an ADR or an outsourced service provider. The purposes for using the data by ADRs are clearly delineated and there are regulatory limits on how long the data can be shared.

Finally, trust in organisations within the CDR ecosystem is increased, because accreditation is a thorough process. This improves market efficiency and allows for easier customer choices. 

Companies that participate as ADRs also find a range of advantages to receiving data as an ADR over screen scraping.

  1. Obtaining data mandated to be shared by the Data Holder is free of charge.
  2. Consistent data models across Data Holders doesn’t just ensure the sharing of customer data; they also significantly reduce the cost of obtaining data from different sources over screen scraping. The countless chaos caused by simple UI changes on the Data Holder side will become history.
  3. Clear SLAs of CDR APIs are mandated for of reliability of sharing data.
  4. There is a roadmap of data types and applicable customer groups as well as for other capabilities within the CDR. The economy-wide commitment for industries that must make data sharing available will create new opportunities for innovation.

The CDR security profile consists in large part of the Financial Grade API (FAPI) standard. FAPI is built on top of the more familiar OpenID Connect, which in turn is built on top of OAuth2.0. It has been designed by the FAPI Working Group (part of the OIDC Foundation) for the specific purpose of securing the FinTech industry.

Fintech services such as aggregation services use screen scraping and store user passwords. This model is both brittle and insecure. This working group aims to rectify the situation by developing a REST/JSON model protected by OAuth./ -FAPI Working Group

There have been some deviations from the FAPI standard within CDR which have been acknowledged by the standards body. The long term intention is to align as much as possible to international standards and to rectify most of these deviations as soon as practicable. Although FAPI provides end users with an advanced level of security when implemented, it still follows a similar user experience to authentication as services they are already familiar with, such as “Sign in with Google” for example. In this way a certain level of customer training is already in place. Instead of “Sign in with Google” the consumer will “Sign in with Bank” and know that their username and password are never handed over to a third party, providing security, confidence and an assurance that no account terms and conditions have been violated.

The advantages of the regulation are largely skewed towards the customers and increase their expectations about technology adoption. They will demand relationships with companies that provide secure, ethical, and modern solutions. In other words, companies that operate as ADRs in the new CDR landscape.


Open Banking Disadvantages Have a Limited Life

In the early days of Open Banking, there are a few downsides, most of which have a limited lifespan. Consumers and business may not be able to share all data types from all organisations just yet, but this will become extinct as the data sharing roadmap for Data Holders progresses. 

There are a few drawbacks for companies to participate in Open Banking as an ADR, including the time, cost, and effort required to become an ADR and maintain required capabilities. This includes internal policies and processes, information security assurances, and meeting the changing technical requirements. Fortunately, there are CDR partners like Adatree to alleviate many of the burdens associated with the technical requirements for accreditation. 

Another drawback is the changing regulation. While the nuances of the regulations and standards are being worked out, there are frequent changes to the customer experience guidelines, technical standards, and CDR rules. These may be challenging to keep up with or difficult to apply for the organisation’s specific use cases. This includes data storage, working with other organisations, de-identification, derived data applications, and more. This requirement to keep up with changes can be bypassed by working with a CDR partner that is focused on CDR compliance requirements, such as Adatree. 

While there are costs, effort, and time required to participate in Open Banking, this can be made easier with experienced CDR partners like Adatree.   

Copy of Advantages, Disadvantages of CDR & Screen Scraping


CUSTOMER EXPECTATIONS SHAPING THE FUTURE OF DATA SHARING

With the promise of better security and quicker data aggregation, CDR prompts the sunrise of digital innovation. As a result, screen scraping, a data sharing method preceding Open Banking, just may see its gloaming. When describing screen scraping in the Review Into Open Banking, Open Banking “should aim to make this practice redundant by facilitating a more efficient data transfer mechanism.” 

Data sharing will evolve because of two push and pull factors: regulatory changes and customer expectations

With the number of companies adopting Open Banking practices growing, so will the number of consumers who will come to expect their third-party providers to offer similar services as their CDR-participant counterparts. The people will gradually come to recognise that there exists an option with secure solutions, privatised credentials, and terms and conditions being honoured. And, needless to say, they certainly won’t see that being the sole wielder of their own data is a bad thing. 

The demands from consumers dictate the highs and lows of the industry, marking the inevitable transition from screen scraping services to Open Banking APIs. Perhaps there will arrive a time when Australian companies will put forth into motion into banning screen scraping like how the UK has done. Perhaps screen scraping will naturally fade away into the backdrop of nugatory, rendered as a technological artifact as Open Banking APIs become commonplace. Whatever the outcome may be, it comes with clarity that the contrivances of APIs will be acknowledged and desired. 

Realistically, there will be a hybrid approach in Australia until one or some of the following happen: 

  1. Open Banking data accessibility surpasses that of screen scraping
  2. Screen scraping is banned in Australia, as has been done in the UK
  3. Customers increase awareness of data transfer methods and demand more control through Open Banking
  4. Regulations change to maintain high technical and security standards but enable a variety of companies to leverage the capabilities of accredited providers 

Regardless of the data sharing method, an environment of ensured technological protection and privacy generates positive impact for all participants in the market and fosters a stronger relationship between companies and their customers.

Screen scraping played a large role in establishing the practice of convenient data access and collection. Alas, just as horse-drawn carriages were replaced by cars and pagers were with cellular devices, screen scraping will hang its hat and recede into the shadows of history. The time has come for consumers to wrest control of their own data and for Open Banking to become the norm.

 

ADATREE’S TURNKEY SOLUTIONS TO ACCELERATE OPEN BANKING

Explore Open Banking data attributes with the Adatree Open Banking Data Sandbox

Stand up a Proof of Concept and test your use case with the Adatree Open Banking Industry Sandbox

Use the Adatree Data Recipient Platform for all technical Data Recipient needs, addressing accessing and leveraging data

DOWNLOAD THIS REPORT

Uncovering the differences between open banking and screen scraping 

 

Exploring the future of regulated and unregulated digital data sharing methods in Australia