1. Audit of Water System Operations: Production CITY Office of the City Auditor
or
DENTON Accountability • Transparency • Integrity • Quality
The City of Denton has established
'' effective processes to monitor and
identify needed water production
infrastructure improvements and to
DENTON report on drinking water quality.
Additional controls over water
treatment chemical deliveries would
further ensure that the City is receiving
_ what is ordered. Similarly, additional
�r guidance over water plant
maintenance work orders could
further streamline the work order
completion process.
Audit of Water System
Operations
Production
Audit Team
City Auditor
Madison Rorschach, CIA, CGAP
Audit Staff
Neeraj Sama, MBA, MS
215 E. McKinney St., Denton, TX 76201 • (940) 349-7228
Audit of Water System Operations: Production September 2021
Table of Contents
Auditat a Glance .........................................................................................................3
Introduction ...................................................................................................................4
Management Responsibility......................................................................................4
Audit Objectives, Scope, and Methodology..........................................................4
Findings & Analysis........................................................................................................6
Water Production Infrastructure Planning Process is Adequate............................ 7
Water Quality Reporting Practices are Generally Effective .................................. 9
Water Treatment Chemicals Secure; Deliveries Not Adequately Reconciled .. 12
Additional Maintenance Work Order Guidance Could Improve Efficiency ..... 14
Safety Assurance Activities Generally Seem Effective......................................... 17
Appendix A: Management Response Summary.....................................................20
Audit Project #: 021 Page 12
Audit of Water System Operations: Production September 2021
Audit at a Glance
Why we did this Audit: What we Found:
Annually, the City produces about This audit generally evaluated the City's water
seven billion gallons of drinking production processes including production
water for its customers. Adequate infrastructure planning & monitoring, water quality
planning and maintenance of reporting, treatment chemical controls, and plant
production infrastructure is critical maintenance. Findings about each of these areas
to ensure drinking water is available are summarized below:
and meets quality regulations. This Production Infrastructure Planning & Monitoring. Water
audit project was included on the production infrastructure must be planned to ensure
City's fiscal year 2020-21 Audit Plan adequate water supply is available to meet peak
as approved by the City Council. water demands. The Water Department appears to
This report details findings and have developed effective processes to forecast
recommendations from phase water supply needs and identify associated water
three of the project. production infrastructure improvements. In addition,
the City currently has water production capacity to
What we Recommend: meet forecasted water demands and is planning to
Recommendation 1 expand this capacity in the next five years.
Consider altering the Daily Sheet Water Quality Reporting. In order to comply with State
recording process to more closely regulations, the City must submit monthly water
align with the Surface Water quality reports. The Water Department's process to
Monthly Operating Report. measure, verify, and report water quality minimizes
Recommendation 2 & 3 the chance for human error. Similarly, the City has
Provide staff written guidelines on adequately licensed staff to meet State
work order completion and requirements. Finally, the City's water system has
prioritization. continuously meet water quality regulations since at
least the beginning of 2018 and has often met water
Recommendation 4 Implement a process to ensure the quality optimization goals.
quantity of chemicals ordered is Treatment Chemical Controls. Furthermore, in order to
delivered. treat raw water, each plant must have an adequate
supply of treatment chemicals. While Water
Recommendation 5 Production has established an effective process to
Maintain an electronic central identify when chemicals need to be ordered, there is
repository for retaining chemical currently no process to ensure the quantity of
delivery bills of lading. chemicals ordered are actually delivered.
Recommendation 6 Plant Maintenance. Water Production has established
Develop a water treatment plant a preventative maintenance program for treatment
safety manual. plant equipment. While this program appears to be
generally effective at identifying and correcting
maintenance issues, some improvements could be
made to how work orders are prioritized and
monitored to improve timeliness.
Audit report translations may be requested by emailing InternalAudit@CityofDenton.com.
Audit of Water System Operations: Production September 2021
Introduction
The Internal Audit Department is responsible for providing: (a) an independent
appraisal' of City operations to ensure policies and procedures are in place and
complied with, inclusive of purchasing and contracting; (b) information that is
accurate and reliable; (c) assurance that assets are properly recorded and
safeguarded; (d) assurance that risks are identified and minimized; and (e)
assurance that resources are used economically and efficiently and that the
City's objectives are being achieved.
The Internal Audit Department has completed a performance audit of the City's
water utility system's production processes. We conducted this performance
audit in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions based on
our audit objectives.
Management Responsibility
City management is responsible for ensuring that resources are managed
properly and used in compliance with laws and regulations; programs are
achieving their objectives; and services are being provided efficiently,
effectively, and economically.
Audit Objectives, Scope, and Methodology
The Internal Audit Department has completed an audit of the City's water
system production processes. This report is intended to provide assurance the
City has adequate controls over water production processes including water
production infrastructure monitoring and planning, water quality reporting,
treatment chemical controls and plant maintenance.
This report is the third phase in an audit project series covering water system
operations. Phase One Rate Structure was published in June 2021 and Phase
Two Distribution was published in July 2021 .
Audit fieldwork was conducted during April, July, and August 2021 . The scope of
review varied depending on the procedure being performed. The following list
summarizes major procedures performed during this time:
I The City of Denton's Internal Audit Department is considered structurally independent as
defined by generally accepted government auditing standard 3.56.
Audit Project #: 021 Page 14
Audit of Water System Operations: Production September 2021
➢ Reviewed documentation to develop criteria including industry standards,
best practices, policies, and procedures;
➢ Developed process narratives to identify current control activities in water
quality reporting, water production equipment maintenance, and water
treatment chemical delivery processes that was certified by Water
Department staff;
➢ Toured the Lake Lewisville and Ray Roberts Water Treatment Plants;
➢ Evaluated annual water demand forecasting methodology to ensure
assumptions seemed reasonable;
➢ Compared planned production infrastructure needs from the 2008 and
2018 Water Master Plans to actual infrastructure improvements and
upgrades and discussed project initiation process with Water Department
staff;
➢ Reviewed Texas Optimization Program Monthly Optimization Reports from
March 2018 through December 2020, the water system's Monitoring Plan,
and the water system's Emergency Response Plan;
Compared water quality data from the February 2020 Surface Water
Monthly Operating Report to Daily Sheets for the same period;
➢ Verified Water Production Operators' licensing aligned with state
regulations and City job description requirements;
➢ Compared bills of lading for 30 water treatment chemical deliveries to
SCADA system data and Daily Sheet entries;
➢ Examined a statistical sample of 352 unscheduled maintenance work
orders2 and a statistical sample of 95 preventative maintenance work
orders3 to determine if maintenance was completed effectively and
timely; and
➢ Interviewed Safety and Water Department staff about safety assurance
activities and examined safety trainings attended by Water Production
staff.
2 This sample size provides with 95 percent confidence that the true population mean is within ±5
percent of the sample estimate.
3 This sample size provides with 95 percent confidence that the true population mean is within
±10 percent of the sample estimate.
Audit Project #: 021 Page 15
Audit of Water System Operations: Production September 2021
Findings & Analysis
The City of Denton's water system currently consists of two water treatment
plants - the Lake Lewisville Water Treatment Plant and the Lake Ray Roberts
Water Treatment Plant -which can treat 30 and 20 million gallons of water a
day respectively. This treated water is then stored in clearwells4 at each plant
before being pumped into the City's distribution system.
Treated water must be adequately pumped to maintain positive pressure, which
prevents contaminants from entering the distribution system. In addition, storage
tanks placed throughout the distribution system help maintain positive pressure
through elevation and help ensure peak water usage demands can be met.
The water production process generally consists of three steps as illustrated in
Figure 1 :
Figure 1: Water Production Process
Treatment . . -
• Raw water • Treated • Treated
is filtered water is water is
and pumped stored
disinfected. through the throughout
distirbution the
system. distribution
system.
This audit generally evaluated the City's water production processes including
water production infrastructures monitoring and planning, water quality
reporting, treatment chemical controls and plant maintenance.
4 Clearwells are storage tanks where filtered water is held as the last stage of treatment to allow
disinfectants to inactivate any remaining pathogens.
5 Based on the City of Denton's 2018 Water Master Plan, water production infrastructure refers to
the City's water treatment plants, clearwell storage, and high service pump stations. Storage
tanks in the distribution system were also included in this definition for the purposes of this audit.
Audit Project #: 021 Page 16
Audit of Water System Operations: Production September 2021
Water Production Infrastructure Planning Process is Adequate
In order to provide effective water service to its customers, a water system must
ensure it has adequate water supply and production infrastructure capacity to
meet daily peak water demands and maintain positive water pressure
throughout the system.
What We Found
• Annually, the Water Department forecasts water production needs as part
of the budgeting process in order to project revenues and inform the capital
planning process. This forecasting methodology appears to be based on
reasonable assumptions and is an effective planning tool.
o This process annually forecasts the City's peak day water usage6 using
the formula shown in Equation 1 , whereby GPCD means gallons per
capital per day and the peaking factor is the ratio of the maximum
day flow to the average day flow.
Equation 1: Peak Day Forecast Equation
GPCD * Population * Peaking Factor = Peak Day
Days in the Year * 1,000,000
o Based on review of the Department's forecast, GPCD estimates
appear to be reasonable based on historical water usage as shown in
Table 1 .
Table 1: Produced Water GPCD Forecast Evaluation (2000-2019)
Year Rain Forecast Historical Estimate Average Historical Actual
Normal Year 158.00 155.41
(32-42 inches)
Drought Year 170.64 165.79
(<32 inches)
Wet Year 145.36 146.65
(>42 inches)
All: 156.59
o Similarly, population is generally estimated to grow about two percent,
which appears to be consistent with the City's growth rate for the last
ten years. In addition, the forecast increased this population estimate
to account for the Hunter and Cole Ranch Development in future
years.
o Finally, the peaking factor used in the forecast is slightly higher than the
actual average peaking factor from 2000-2019.
6 Peak day water usage is the maximum volume of water used in a 24-hour period within a given
year.
Audit Project #: 021 Page 17
Audit of Water System Operations: Production September 2021
• In order to identify water production capacity improvements needed in the
near-term, the Water Department periodically contracts with a consultant to
develop a Water Master Plan. As part of this process, the consultant updates
a hydraulic model of the City's water system, which the City uses to
evaluate the impact of smaller developments on the water system.
o The most recent Water Master Plan was completed in 2018 and
recommended several production infrastructure projects be
completed between 2019 and 2043. The current status of each of
these projects is shown in Table 2.
Table 2: Water Production Infrastructure Recommendations
Project Complete By Status
Riney Road Booster Pump Station 2023 In Design
30.0 MGD RR Water Treatment Plant Expansion 2028 Planned
2.0 MG State School Elevated Storage Tank 2043 Under Evaluation
Southwest Pump Station Improvements 2043 Planned
o In order to ensure that recommended improvements are still required,
the Water Department contracts with an engineering consultant to
perform a hydraulic analysis before initiating a project to ensure the
improvement is actually needed at the forecasted size.
o Similarly, the Water Department intends to update the Water Master
Plan and associated water system model every five years.
• The City currently has the combined water production capacity to meet
projected drought year peak day demands until 2024.
o That being said, the Ray Roberts Water Treatment Plant does not
always have the ability to meet average day water demand alone
as shown in Figure 2. While it is unlikely Ray Roberts will be the City's
only operational treatment plant, the impacts of being unable to
meet water demands can be substantial.
Figure 2: Daily Water Demand & Treatment Plant Capacity
35
Q 30
25
p 20
o p 15 Ray Roberts Capacity
o10
5 Lake Lewisville Capacity
0 Average Daily Demand
6A o1 J)A J5� of o� of of
1`0 160 �P
OoGo
Audit Project #: 021 Page 8
Audit of Water System Operations: Production September 2021
o Water Production is in the process of preparing the Ray Roberts plant
to be rerated by the Texas Commission on Environmental Quality.
Rerating this facility should help mitigate the risk of not meeting the
average day demand alone as the plant's production capacity will
increase.
o In addition, the 2018 Water Master Plan recommended expanding the
Lake Ray Roberts Plant's capacity by 30 million gallons a day. This
expansion is currently included in the Water Department's five-year
capital improvement plan which would further mitigate this risk.
Why It Matters
Providing adequate water supply to its customers is critical for the City to ensure
public health. The Water Department appears to have developed effective
processes to forecast water supply needs and identify associated water
production infrastructure improvements. In addition, the City generally has water
production capacity to meet forecasted water demands and is planning to
expand production capacity in the next five years.
Recommendation: None.
Water Quality Reporting Practices are Generally Effective
The City of Denton's water system is required to meet water quality regulations
set forward in the Texas Administrative Code Chapter 290. Under this chapter, all
public water systems in Texas must submit monthly water treatment data to the
Texas Commission on Environmental Quality. This chapter also requires public
water systems to employ a minimum number of licensed water system
operators.
What We Found
• Based on a review of the Lake Lewisville and Ray Roberts plants' water
quality reports from January 2018 through December 2020, the City has
continuously met the Environmental Protection Agency's water quality
regulations.
o In addition, Water Production began participating in the Texas
Optimization Program in March 2018. This program recognizes water
systems that meet optimization goals set by the United States
Environmental Protection Agency and is run by the Texas Commission
on Environmental Quality. Table 3 summarizes each plants
performance as it relates to these optimization goals.
Audit Project #: 021 Page 19
Audit of Water System Operations: Production September 2021
Table 3: Plant Optimization Goal Performance
Percent of Months Meeting Goal
Lake Lewisville Ray Roberts
Filter Goals 100% 94%
Backwash Goals 100% 100%
Settled Water Goals 85% 97%
Months Participating: 34 33
• Water Production has developed a semi-automated process to measure
and report on water quality, which minimizes the chance for human error.
o Water quality readings are automatically taken periodically by the
Water Department's supervisory control and data acquisition - or
SCADA - system using instrumentation7 within the water treatment
plants. These readings are then imported from the SCADA system into
the required Surface Water Monthly Operating Report.
o In addition, water quality metrics are recorded throughout each day
by Water Production Operators in Daily Sheets. These Sheets are then
used at the end of each month to identify any discrepancies in the
Surface Water Monthly Operating Report.
o Based on a comparison of the Surface Water Monthly Operating
Report and Daily Sheets from February 2020, there was not a significant
difference for about 93 percent and 91 percent of the active plant
days for the Lake Lewisville and Ray Roberts plants respectively.$
o While these Sheets generally provide enough information to identify
discrepancies, recording Daily Sheet turbidity9 readings on even hours
could increase comparability between the Daily Sheets and the
Surface Water Monthly Operating Report.
• As required by the Texas Administrative Code, the Water Department has
developed and submitted a Monitoring Plan to the Texas Commission on
Environmental Quality that demonstrates that the water quality monitoring
performed by the system is consistent with regulatory requirements.
• The City annually publishes a Consumer Confidence Report to all its
customers as required by the Environmental Protection Agency, which
provides information on any regulated contaminants found in the local
7 Means flow meters, pH meters, pressure meters, turbidity meters, and more.
8 Whereby a significant difference in the daily average was a difference ±0.005 nephelometric
turbidity units.
9 Turbidity is the measure of relative clarity of a liquid and is the primary measurement used in
water quality since particles in water provide attachment places for metals, bacteria, and other
pollutants.
Audit Project #: 021 Page 1 10
Audit of Water System Operations: Production September 2021
drinking water. The information produced in this report is based on tests run
by the Texas Commission on Environmental Quality. This report is also known
as the annual drinking water quality report.
• Based on a review of the City's Water Production Operator licenses, the
Water Department has adequate water system operator licenses as
required by state regulations to operate its water treatment plants.
o In addition, the City of Denton has established different levels of Water
Production Operators which are distinguished by the minimum required
level of water system operator licensing. Water Production staff
licensing is tracked by Water Production management, which is
required to report on staff licensing to the Texas Commission on
Environmental Quality periodically.
o Based on a comparison of each Water Production Operators'
associated job descriptions and current licensing, all 18 Water
Production Operation staff meet minimum job description licensing
expectations as shown in Table 3; however, five have not obtained
licenses in accordance with the Department's progression plan.
Table 4: Water Production Operator Licensing Summary
Operator Level Staff Minimum Career Licenses
Requirement Progression Step A B C D
1 6 None C in Three Years 0 0 2 4
II 8 C Upon Hire B in Three Years 0 0 8 0
III 2 B Upon Hire NA 1 1 0 0
Foreman 2 A Upon Hire NA 2 0 0 0
Why It Matters
The production and distribution of drinking water in the United States is highly
regulated by the Federal and state governments. Non-compliance with
licensing and reporting requirements could lead to significant penalties and
public health impacts. Based on this review, Water Production appears to have
established effective processes to report on water quality and track operator
licensing. Slight adjustments to the Daily Sheet process might further improve
their effectiveness as a control.
In addition, Water Production's participation in the Texas Optimization Program
is considered a best practice. While the City's water treatment plants are not
always meeting these optimization goals, they have continuously met water
quality regulations established by the United States' Environmental Protection
Agency.
Audit Project #: 021 Page 1 11
Audit of Water System Operations: Production September 2021
Recommendation:
1. Consider taking Daily Sheet turbidity readings each even hour instead of
each odd hour to more closely align with turbidity readings reported in the
Surface Water Monthly Operating Report.
Water Department Comments: It is believed that this recommendation may
not result in al numbers matching from the manual readings to the SCADA
readings as the SCADA system records many more points than our staff
reports manually. However, Water production has implemented even hour
recording at LLWTP. The RRWTP was already recording the readings on the
even hours.
Water Treatment Chemicals Secure; Deliveries Not Adequately
Reconciled
The City has contracted with ten vendors to supply chemicals needed for water
treatment at its plants. In general, these chemicals are used to reduced turbidity
and disinfect water. Best practices suggest that the following controls should be
in place to ensure chemical inventories are properly safeguarded:
➢ Physically secure inventory so only authorized personnel have access;
➢ Record inventory usage; and
➢ Count all incoming inventory to ensure what is received is what was
ordered.
What We Found r� I
• Chemicals used in the water treatment
process are safeguarded on site at each
water treatment plant, which are fenced in
and require appropriate authorization to
enter. In addition, each storage tank is
locked and keyholders are tracked.
r
"
• Water Production staff have developed an
effective process to monitor chemical levels
and order new inventory when needed.
o Bulk chemical levels are automatically
recorded by the Department's Picture 1: Chemical Storage Tank
supervisory control and data acquisition
- or SCADA - system regularly. Twice each day, Water Production
Operators record chemical tank levels in their Daily Sheet. These levels
Audit Project #: 021 Page 1 12
Audit of Water System Operations: Production September 2021
are reviewed by the Water Production Operations Supervisor to
determine if additional chemicals need to be ordered.
o Based on review of a judgment sample of Daily Sheets recorded on
five chemical deliveries days,10 chemical deliveries captured by the
SCADA system were reflected in chemicals levels recorded in the Daily
Sheets.
• Water Production staff do not have a process to ensure that the quantity of
chemicals ordered is delivered appropriately.
o As chemicals are being delivered, water production staff ensure that
the chemical is the correct type based on color and specific gravity
before allowing the offloading process to being. In addition, chemical
storage tanks must be unlocked by authorized Water Production staff
and all deliveries are observed during the offloading process.
o Based on a review of thirty chemical deliveries, chemical levels on five
days recorded by SCADA were significantly different from what would
be expected based on the ordered quantity. In addition, SCADA
readings could not be obtained for 19 of the delivery days due to a
SCADA system crash happened in the past. It should be noted that
SCADA readings must be converted from feet to pounds or gallons to
compare ordered and delivered quantities.
• According to Water Production staff, they physically retain all bills of lading;
however, of a sample of 30 orders the Department could not provide bills of
lading for nine sampled orders.
o Based on discussion with Water Department management, this issue
may be due to a staffing vacancy caused by the COVID-19
Pandemic.
Why It Matters
Chemical inventory is critical to water production operations to ensure that raw
water is adequately treated. For this reason, chemical inventory levels must be
adequately maintained and safeguarded. While chemicals appear to be
adequately safeguarded by each water plants physical security, there is
currently no process to reconcile delivered chemicals to those ordered.
Implementing a reconciliation process will provide assurance that not only is the
City appropriately spending its money, but also that chemicals will be available
10 Due to a process change, only five of 30 reviewed Daily Sheets had chemical inventory levels
recorded.
Audit Project #: 021 Page 1 13
Audit of Water System Operations: Production September 2021
when needed. In addition, bills of lading should be retained in order to provide
support for the chemicals received.
Recommendations:
2. Implement a process to ensure the quantity of chemicals delivered matches
the quantity ordered. It may be necessary to create and adopt formulas with
acceptable margins of error to convert delivery amounts to those reflected
in the SCADA system.
Water Department Comments: The Department has started to use Infor EAM's
work priority field. The field allows the selection of high, medium, medium low,
medium high, and low. Their priority level will be designated by the manager
or supervisor responsible for the work request. Any items related to safety,
compliance or environmental impact will receive high priority while items
related to building maintenance or aesthetics will receive a medium to low
priority. Written guidance will be provided as part of Recommendation #3.
3. Maintain a central repository for keeping electronic copies of bills of lading to
aid the chemical delivery reconciliation process. In addition, determine how
long bills of lading should be retained per the appropriate Texas State
Records Retention Schedule.
Water Department Comments: Will develop written guidance on how to
complete work orders. This guidance will also include Recommendation #2.
Additional Maintenance Work Order Guidance Could Improve
Efficiency
Over time, the performance of water production equipment eventually
degrades. In order to ensure each water treatment plant continues to be
capable of producing drinking water, this equipment must be adequately
maintained and repaired.
Timely and appropriate maintenance of water production equipment also helps
ensure this equipment meets its useful life and water production is not impacted.
In general, there are three types of maintenance programs as defined below:
v Reactive Maintenance - no actions or efforts are taken to maintain
equipment to ensure expected life is reached;
v Preventative Maintenance - actions performed on a time- or machine-
run-based schedule that detect, preclude, or mitigate degradation with
the aim to extend the useful life of equipment; and
Audit Project #: 021 Page 1 14
Audit of Water System Operations: Production September 2021
s Predictive Maintenance - measurements that detect the onset of system
degradation, thereby allowing casual stressors to be eliminated prior to
deterioration.
Each of these program types has its own advantages and disadvantages,
though it should be noted preventative and predictive maintenance programs
still include reactive maintenance activities - these activities just make up a
smaller percentage of work performed.'
What We Found
• Water Production has generally established an effective preventative
maintenance program, where about 20 percent of maintenance activities
are reactive - or unscheduled maintenance - as shown in Figure 3.
o While additional cost savings could be observed in the long run from
increasing predictive maintenance activities, this type of program
requires large, short-term investment in technologies to monitor
equipment condition. Due to the size of the City, this type of large,
technological investment may not be necessary at this time.
Figure 3: Plant Work Orders by Type
Non-Maintenance
Unscheduled 6%
Maintenance
19%
Preventative
Maintenance
75%
• Water Production utilizes a work order management system to assign
maintenance work to staff. This process appears to effectively ensure work
orders are assigned to appropriate staff.
o Most work orders are automatically generated as part of the
preventative maintenance program. In addition, unscheduled
maintenance activities can be reported by all Water Production staff.
>> Typically, a maintenance program is considered reactive if more than 55 percent of
maintenance activities are reactive.
Audit Project #: 021 Page 1 15
Audit of Water System Operations: Production September 2021
These work orders are then reviewed by Water Production
management to ensure they are necessary before being assigned.
• Based on review of a statistical sample of 95 preventative maintenance work
orders, about 17 percent were not completed in a timely manner.12
o This issue appears to be due to delays in the assignment of
preventative maintenance work orders s rather than how quickly staff
are completing them.
• Based on review of a statistical sample of 352 unscheduled maintenance
work orders, it was unclear if the maintenance issue had been resolved
based on the comments entered for about 10 percent. For an additional
three percent of work orders, the comments indicated that the maintenance
issue was not resolved.
• According to Water Production staff, work orders are generally prioritized
informally as part of the daily staff meeting. That being said, there is no clear,
written guidance around how work orders should be prioritized based on the
work needing to be performed.
• Similarly, the timeliness of unscheduled work orders could not be determined
as there is no written guidance around how quickly these work orders should
be completed.
o According to staff, Water Production management meets every two
weeks to discuss work needing to be completed and writes comments
on outstanding work orders to check up on their status.
Why It Matters
An effective work order system is critical to ensure that maintenance work for
water treatment plants is completed effectively and timely. Water Production
staff appear to have established a preventative maintenance program, which
generally appears to be effective at ensuring critical plant equipment is
monitored for maintenance needs. That being said, a significant portion of
preventative maintenance work orders are not completed in a timely manner.
Furthermore, while unscheduled maintenance generally appears to be
completed effectively, some documentation limitations make it difficult to
determine if the issue was corrected. Moreover, it is unclear if unscheduled
12 For preventative maintenance purposes, a work order was timely if it was completed before
the next preventative maintenance work order in that schedule was issued.
Audit Project #: 021 Page 1 16
Audit of Water System Operations: Production September 2021
maintenance is being completed in a timely manner as there are no guidelines
around how quickly these should be completed.
While having periodic management meetings to check on the status of
outstanding work orders is appropriate, without documentation of this follow-up
and staff's response, it is unclear if maintenance delays are reasonable. Similarly,
developing guidelines for staff on how to prioritize work orders could further
improve timeliness as staff would have a clear reference for which work should
be completed first.
Recommendations:
4. Develop guidelines for Production staff on how to prioritize work orders by
type.
Water Department Comments: Staff implemented chemical delivery tracking
on operations paperwork. In addition, all vendors will be contacted to
ensure they provide certified weight tickets as required by our contract
specification.
5. Provide staff written guidance on how to complete work orders to ensure all
needed information is included. Ensure that management follow-up on
outstanding work orders are included in the comments.
Water Department Comments: This was the procedure until we lost our
administrative assistant to the voluntary separation program. The
department was told we did not need to keep these documents. The
department has reinstated the practice of keeping these documents
electronically,
Safety Assurance Activities Generally Seem Effective
The American Water Works Association's best practices recommend that water
utilities establish a health, safety, and environment program. In general, this kind
of a safety program includes the policies, procedures, training, personal
protective equipment, and lifesaving equipment needed to perform the work
safely.
What We Found
• Water Production staff regularly attends safety training organized by the
Safety Department including the Federal Emergency Management Agency's
required Incident Command System courses, defensive driving, heavy
equipment awareness and confined space training.
Audit Project #: 021 Page 1 17
Audit of Water System Operations: Production September 2021
o As part of the Safety Department's initiatives a safety committee
consisting of Water and Safety Department staff has been formed that
meets monthly to discuss safety concerns, procedures, and initiatives.
o Safety Department staff also meet regularly with Water Department
staff to discuss any safety concerns and get feedback. In addition, the
Safety Department is planning to begin periodic inspections of the
City's water treatment plants.
• Water Production management also supplements this training with the
Occupational Safety and Health Administration's hazardous waste
operations and emergency response training and hazard communication
training. These trainings are specifically focused on responding to hazardous
materials, including those that are kept on site at the City's water treatment
plants.
o According to Water Production's records, all 31 staff have received the
hazard communication training, and nine have undergone the
hazardous waste operations training. Based on discussion with Water
Department staff, the hazardous waste operations training must be
performed in person and so roll out has been delayed due to the
COVID-19 Pandemic.
o In addition, Water Production staff meet periodically to have "tailgate"
meetings, during which they discuss the day's activities and go over
certain safety procedures.
• Water Production has developed some written safety guidance to aid staff
in the event of an emergency and as they complete their work.
o As required by the 2018 America's Water Infrastructure Act, the Water
Department has developed an Emergency Response Plan that details
emergency response procedures for several emergency situations
including extreme weather events, source water contamination,
cyberattacks, active shooters or other physical security threats, and
power loss.
o In addition, Water Production has formalized a Hazard Communication
Program, which is intended to ensure that employees have adequate
information to identify hazardous substances and prevent illness if
exposed, and to clarify who is responsible for maintaining the Program.
o That being said, Water Production does not have a comprehensive
safety manual that details safety practices and standards needed
throughout the plant.
Audit Project #: 021 Page 1 18
Audit of Water System Operations: Production September 2021
Why It Matters
Education and specialized safety training are important tools for making Water
Production staff aware of the hazards and safety procedures involved in their
jobs. If any safety incident happens on a water plant, it may result in injury to
City staff, financial losses, and potential drinking water hazards.
The Water and Safety Departments are providing knowledge and education to
Water Production staff through training. Development of a comprehensive
safety manual describing the safety standards and procedures needed on a
water treatment plant would help provide clear, written guidance to Water staff
on how to adhere to safety measures that will eventually be used by the Safety
Department to perform safety inspections.
Recommendation:
6. Incorporate water treatment plant safety standards and practices into a
Water Department safety manual in consultation with the Safety Department
for staff guidance.
Water Department Comments: Water Production will work with Safety to
develop the manual. The outline is currently underdevelopment.
Safety Department Comments: The Safety Department concurs with
Recommendation #6.
Audit Project #: 021 Page 1 19
Audit of Water System Operations: Production September 2021
Appendix A: Management Response Summary
The following summarizes the recommendations issued throughout this report. The
auditors found that staff and the Department were receptive and willing to make
improvements where needed. Management has provided their response to each
recommendation.
Consider taking Daily Sheet turbidity Expected
readings each even hour instead of each Completion:
1 odd hour to more closely align with turbidity Partially Completed
readings reported in the Surface Water Concur
Monthly Operating Report.
Water Department Comments: It is believed that this Responsibility:
recommendation may not result in al numbers matching from Plant
the manual readings to the SCADA readings as the SCADA Managers
system records many more points than our staff reports
manually. However, Water production has implemented even
hour recording at LLWTP. The RRWTP was already recording the
readings on the even hours.
Develop guidelines for Production staff on Expected
2 how to prioritize work orders by type. Concur Completion:
10/30/2021
Water Department Comments: The Department has started to Responsibility:
use Infor EAM's work priority field. The field allows the selection Plant
of high, medium, medium low, medium high, and low. Their Managers
priority level will be designated by the manager or supervisor
responsible for the work request. Any items related to safety,
compliance or environmental impact will receive high priority
while items related to building maintenance or aesthetics will
receive a medium to low priority. Written guidance will be
provided as part of Recommendation #3.
Provide staff written guidance on how to Expected
3 complete work orders to ensure all needed Concur Completion:
information is included. 10/30/2021
Water Department Comments: Will develop written guidance Responsibility:
on how to complete work orders. This guidance will also Plant
include Recommendation #2. Managers
Implement a process to ensure the quantity Expected
4 of chemicals delivered matches the Concur Completion:
quantity ordered. Completed
Water Department Comments: Staff implemented chemical Responsibility:
delivery tracking on operations paperwork. In addition, all
Audit Project #: 021 Page 1 20
Audit of Water System Operations: Production September 2021
vendors will be contacted to ensure they provide certified Plant
weight tickets as required by our contract specification. Managers
Maintain a central repository for keeping Expected
5 electronic copies of bills of lading to aid the Concur Completion:
chemical delivery reconciliation process. Completed
Water Department Comments: This was the procedure until we Responsibility:
lost our administrative assistant to the voluntary separation Plant
program. The department was told we did not need to keep Managers
these documents. The department has reinstated the practice
of keeping these documents electronically,
Incorporate water treatment plant safety Expected
standards and practices into a Water Completion:
6 Department safety manual in consultation Concur 7/31/2022
with the Safety Department for staff
guidance.
Water Department Comments: Water Production will work with Responsibility:
Safety to develop the manual. The outline is currently under Water
development. Production
Safety Department Comments: The Safety Department concurs Superintendent
with Recommendation #6. & Director of
Safety
Audit Project #: 021 Page 121