Data Visualization and Performance Audits

By Ms. Smita Gopal, Sr. Dy. Accountant General, Office of the Principal Accountant General(G&SSA, Bengaluru

1. INTRODUCTION

This paper examines the use of Data visualization in performance audits drawing from the experience of a recent performance audit conducted in Karnataka on the Conservation and Ecological Restoration of Lakes. The objective of this paper is to explain how the use of Data Visualization to represent various audit findings enhances the presentation and readability of Performance Audit Reports.

2. ABOUT THE PERFORMANCE AUDIT CONDUCTED IN KARNATAKA

The performance audit on “Conservation and ecological restoration of lakes under the jurisdiction of Lake Development Authority and Urban Local Bodies” was conducted to assess the effectiveness of the initiatives taken by various agencies involved in conservation and rejuvenation of the lakes in urban and semi-urban areas.

The performance audit was conducted jointly by the offices of Principal Accountant General (General & Social Sector Audit) and Principal Accountant General (Economic Revenue Sector Audit) for the period 2009-14.

It covered the activities relating to conservation and restoration of lakes in Revenue Department, Forest Department, Urban Development Department (UDD), and Fisheries Department. The role of various implementing agencies under these departments namely, LDA, Bruhat Bengaluru Mahanagara Palike (BBMP), Bengaluru Development Authority (BDA), two City Corporations (CCs) (Belagavi and Hubballi-Dharwad), Karnataka State Pollution Control Board (KSPCB) and Bengaluru Water Supply and Sewerage Board (BWSSB) has also been covered in the performance audit.

The performance audit covered 56 lakes selected by adopting simple random sampling method. The test-checked lakes include 13 lakes of BBMP; 19 lakes of BDA, three lakes of Belagavi and 10 lakes of Hubballi-Dharwad CCs.

Besides, lakes under National Lake Conservation Plan (NLCP)¹ (six lakes including two in Bengaluru) and National Wetland Conservation Plan (NWCP)² (two lakes) and three lakes (out of state grants) under the control of LDA were also selected.

3. POTENTIAL FOR DATA VISUALIZATION IN THE PERFORMANCE AUDIT

There were various entities and issues covered. The main issues or areas of audit included inter alia;

• Survey and demarcation of lakes (Revenue Department),

• Water quality, pollution in lakes ( KSPCB, LDA, BBMP, BDA, ULB )

• Sewerage related issues ( BWSSB and ULBs)

• Issues regarding Biodiversity (KSPCB, LDA, BBMP, BDA, ULB )

• Execution of Works in lakes ( LDA, BBMP, BDA, ULB)

• Governance , Accountability, Institutional Mechanism ( All entities)

• Planning and coordination amongst various entities ( All entities)

• Financial Management related issues( All entities)

Under issue cited above, there were several audit findings. Before data visualization was attempted, most of the audit findings were entirely in the narrative and in some cases supported by further information in the annexures or tables within the finding itself.

4. ISSUES IN DATA VISUALIZATION

When the team first sat down for attempting data visualization, it was clear that there were was sufficient potential for doing the same. As the team got into the details of this exercise, the data visualization charts went through several rounds of corrections before they could be finalized. The issues faced by the team are given below:

4.1. Deciding the findings to be depicted through data visualization

The draft performance audit report contained several findings based on the test checked lakes. The idea was to cull out those findings which would give an idea about the entire statistically selected sample and therefore generalize on the general status of lakes in Bangalore and Karnataka. This was not possible, since in some cases, the finding was not about all the test checked cases. For example, in the first draft, the audit finding on “Guard rooms”, “Inadequate islets and outlets of lakes” and “Boat jetties” were about only some and not all the selected lakes. Such issues could not be used in data visualization.

In fact, this exercise also enabled the teams to appreciate which findings were more significant and material and eventually some of those findings which were applicable only to a few lakes were either removed altogether from the report or taken to the last section which contained findings on test checked lakes.

There were also some findings which were not about the test checked lakes, but about the administration of lakes in general such as “Inspections and Monitoring” or "Grievance Redressal Mechanism”.

Thus those findings could be best depicted visually where there was complete data in respect of the test checked sample. Those findings which by their very nature were such that they could not be visually depicted had to be excluded from such an exercise.

4.2. Which Chart? One Chart or Many Charts?

As there are many ways in which graphical techniques can be used to explain the audit findings, it was important that the graphical representation of the audit findings was not too cluttered and hard to comprehend. The team found that if there were too many issues in a single representation, it would confuse the reader and then defeat the purpose of making the report more presentable.

4.3. Use of Colours in the Chart

The reason a chart is more presentable is because of the visual impact it creates. Thus care should be taken to maintain some consistency in the use of colour in the charts.

The main type of charts used in the Report was one where the X axis depicted the number of lakes and the Y axis captured the various attributes. In respect of each attribute, the status in the sampled lakes was shown by red green and orange bars which depicted the number of lakes. In the initial attempt, it was seen that there was no logic to the colours used. For example-

1. Lakes free of Pollution(Yes/No)

2. Lakes with invasive species like water hyacinth(Yes/No)

In order to overcome it, all attributes/ questions in chart were framed in a uniform manner so that if there was a positive Audit finding, it could be indicated in green colour and a negative finding could be indicated in Red Colour.

For example -

1. Lakes free of Pollution(Yes/No)

2. Lakes without invasive species like water hyacinth(Yes/No)

It can be seen that the attribute of the lake is framed in such manner the green colour indicates a positive attribute and the Red Colour indicates a negative attribute.

Moreover, the colour sequence should also be followed uniformly.

4.4. Arranging the audit issues in a logical manner

It is also important that the sequence of issues in a chart follow some pattern. Either it should be in the order of the importance of the issue or it should be arranged in the sequence of the process. In case, the issues in the chart are also elaborated in the report, it should be ensured that the sequence of those findings in the report is in the same sequence as given in the chart.

4.5. Basic Attributes of a Chart should not be Missing

It must also be ensured that basic attributes of the chart are not missing. The Chart should have a title. The x and y axis should also contain proper titles. The Legend (or the key) must containthe list of the variables appearing in the chart and an example of their appearance. This information allows the data from each variable to be identified in the chart.The size of the text and numbers appearing in the charts should not be too small.

Eventually it was easy to overcome these issues. One example was applying the traffic signal approach for the use of colours. The Attributes/ Questions in chart were framed in such a manner that thePositive Audit finding would be indicated in green while a negative response would be indicated in red.

5. BENEFITS OF DATA VISUALIZATION

5.1. Reduction in Length

The first draft attempted was elaborate and quite lengthy. There was a conscious effort to provide as many details as possible. After data visualization was attempted, the length was substantially reduced as several findings had been correlated and the charts enabled the report to be made more concise.

5.2. Neater depiction of audit findings- Use of Annexures

In the first draft, several annexures were included in support of the audit findings. These were reviewed and some of them were removed and their finding was replaced with simpler reading charts. This included two anexures on ‘Details of surveys conducted by Revenue Department’ and ‘Grant of lake land’, the findings of which were put in the chart on Issues on effectiveness of survey and status of encroachments in the test-checked lakes.

5.3. Inclusion of more Details by way of Chart

One of the findings in the first draft was an important issue relating to core (ecological works such as desilting, creation of wetlands etc) and non-core (non-ecological works such as civil works of making guard rooms etc) works. The original para did not give the lake wise details on the break-up between the core works and the lesser desirable non-core works. However, data visualization of the finding enabled the lake wise details to be incorporated in the report.

The chart used is given below:

Chart 6. Provision for core and non-core works as per DPRs/estimates.

5.4. Greater Impact of audit findings- More Convincing

The use of data visualization to depict findings enables all the findings related to a particular audit area to be quantitatively summarized at one place. It provides a bird’s eye view of that entire audit area.

For example, there was a chapter on biodiversity which contained findings in respect of the test checked lakes on issues such as Non-preservation of foreshore area of the lake, Inadequate inlets and outlets of lakes, Absence of database on Inventory of Species, Inadequate efforts to tackle invasive species and Lack of initiative to preserve Buffer Zone. In each observation, crux of the finding was that biodiversity as a subject had been largely ignored by the implementing agencies who had evidently done very little in the area of preserving biodiversity of lakes. However, in order to arrive at this conclusion, the reader would need to read the entire chapter (8 pages) which also happened to contain a few other findings like afforestation etc which were not on all the test checked lakes.

In the final draft, this was remedied by incorporating a chart right in the beginning of the chapter titled “Biodiversity Issues” which covered five key issues. A reader could easily ascertain from one glance at the chart that biodiversity issues had not been given due importance.

5.5. Overcome the issue of “Convenient Reporting

Certain times, audit findings are reported without reference to the audit sample. For example, in the initial draft, the audit finding stated that “during the joint physical verification and review of detailed project reports Audit came across many cases of possible encroachments other than those where the Revenue Department had itself made the grant. These cases are listed in the Appendix. Photographs given below underscore this infraction.”

Thus, it can be seen that when words such as "many" or "few" or "some", they do not refer to the entire sample of the test checked items, which was lakes in this case. For the reader of the report, it will not be clear whether data on all test checked entities is even available or not or whether the finding is restricted to the test checked lakes.In the first draft, many of the findings were not in respect of all the test checked lakes. While, in some cases, it was because it was not possible to obtain data on all the test checked lakes, in some cases although data had been available but due to various exigencies, it was not mentioned.

The use of data visualization ensures that reference is made to the entire sample and findings are thus quantified enabling the reader to easily determine the materiality of the issue.

6. WAY FORWARD

6.1. Data Visualization to be planned from Detailed Audit Proposal Stage/ PA guidelines

At the time of preparation of the PA guidelines, teams should devote some time planning what findings could be graphically depicted. This will enable to the teams to call for and obtain data that could be quantified. Planning for data visualization would be particularly useful where there is more than one team conducting the audit.

6.2. Training for Data Visualization

It is necessary that the teams conducting the performance audit should have basic excel skills and be able to represent findings graphically. This would enable the team to determine the more appropriate chart to capture the relevant data and bring out the crux of the issue.

Reliance on other “data visualization experts” may dilute the impact of the findings especially if the chart is unable to capture the data capturing the crux of the issue.

The office should also train a few persons who have advanced data visualization expertise and so that their inputs could be used by all Performance Audit team.

6.3. Examining potential for data visualization during Mid-Term Review

The Mid-term review of the Performance Audit should be used as an opportunity to examine the potential of findings that can be visually depicted. A review should be done of what was planned in the beginning of the audit and what data has been collected against it. Uniformity in the manner in which data is being collected could also be seen and any mid course interventions could be made to avoid scrambling for data not collected towards later stages of the audit.

6.4. Peer Reviewing use of data visualization

Lastly it is important that the graphs and charts proposed to be incorporated in the report and peer reviewed by persons who have not been associated with the performance audit. This is essential to ensure that the basic purpose of data visualization to make the report more presentable and convincing is achieved. The feedback from non-associated persons would go a long way in improving the quality of the way in which data has been depicted.

The feedback from non-associated persons would go a long way in improving the quality of the way in which data has been depicted.

1. 1. A centrally sponsored scheme exclusively aimed at restoring the water quality and ecology of lakes in urban and semi-urban areas
2. 2. GoI Scheme for conservation of wetlands to benefit the local communities and biodiversity