STATE OF HAWAII
BOARD OF EDUCATION
Queen Liliuokalani Building
1390 Miller Street, Room 404
Honolulu, Hawaii 96813
Thursday, November 1, 2018
Catherine Payne, Committee Chairperson
Dwight Takeno, Committee Vice Chairperson
Brian De Lima, Esq.
Christina Kishimoto, Superintendent
David Brookshire (“Brook”) Conner, Assistant Superintendent and Chief Information Officer, Office of Information Technology Services
Denise Yoshida, Director, Internal Audit Office
Yi Chen, Audit Specialist, Internal Audit Office
Alison Kunishige, Executive Director
Kenyon Tam, Board Analyst
Regina Pascua, Board Private Secretary
Irina Dana, Secretary
I. Call to Order
The Audit Committee (“Committee”) was called to order by Committee Chairperson Catherine Payne at 11:31 a.m.
II. *Public testimony on Audit Committee (“Committee”) agenda items
Committee Chairperson Payne called for public testimony. There was no public testimony at this time.
III. Approval of Meeting Minutes of May 3, 2018 and September 6, 2018
Committee Member Brian De Lima stated the Committee should reconsider the meeting minutes of May 3, 2018, as the redline amendments that the Committee approved at its September 6, 2018 meeting were not posted on the Board of Education’s (“Board”) website.
ACTION: Motion to rescind the approval of the Audit Committee Meeting minutes of May 3, 2018 as circulated on September 6, 2018 and approve the Audit Committee Meeting minutes of May 3, 2018 without amendments (De Lima/Takeno). The motion carried unanimously with all members present voting aye.
ACTION: Motion to approve the Audit Committee Meeting minutes of September 6, 2018 (De Lima/Takeno). The motion carried unanimously with all members present voting aye.
IV. Discussion Items
A. Presentation of the Department of Education’s Internal Audit Plan—Quarterly Update through September 30, 2018
Denise Yoshida, Director, Internal Audit Office (“IA”), stated that the Department of Education’s (“Department”) internal audit plan quarterly update covers the period between July 1, 2018 and September 30, 2018. She reviewed an executive summary of assurance projects; consulting, monitoring, and fiscal reviews; and other projects. Yoshida highlighted that IA completed the Student Information System Post-Implementation Review. IA also performed two student activity fund reviews and two school monitoring reviews. She stated that IA continued to coordinate and monitor the assignments of the cases received from the Fraud and Ethics Hotline and performed several unanticipated fiscal reviews. Yoshida detailed that IA followed up with the Office of the Superintendent on the status of the standard practices (“SPs”) project. Currently, the SPs project is in various stages. Most offices are in the process of drafting and updating their SPs, and some offices have had their SPs reviewed by the Department of the Attorney General and are now making corrections. An internal worksite has been established on the Department’s intranet to automate the process as well as provide standardized templates. Yoshida highlighted that the Department has updated 176 SP drafts on the intranet using the new templates. Upon approval, the Department will post them to its public website. Yoshida stated that this quarter, IA began coordinating and assisting the certified public accounting firm KKDLY LLC with the Department’s Annual Financial and Single Audit for the fiscal year ending on June 30, 2018.
Yoshida reviewed a summary of proposed changes to the audit plan. She noted that IA proposed moving the Post-Investigation Decision Making Process Audit to its watch list due to the issuance and implementation of the new investigation manual. Yoshida detailed that during the initial stages of meetings with the Office of Talent Management, IA was notified that the manual was almost complete. Once the manual’s distribution and training occurs, it will give the field time to implement the process, and IA will include it in a future audit plan. Yoshida reviewed the management action item dashboard summary and the status of IA observations and recommendations from previously issued reports as reported to management. She highlighted that there are no critical areas for IA to draw attention because management is within target due dates.
Yoshida reviewed the Fraud and Ethics Hotline summary. She reviewed the status of all open hotline cases as of September 30, 2018, and noted that IA closed a couple of these cases since it generated its report. Yoshida reviewed case statuses, the summary of cases by fiscal year, and outstanding cases from prior years. She noted that the Fraud and Ethics Hotline usually receives 200 calls or intakes per year. It received 42 cases in the first quarter and a flood of calls in the beginning of the second quarter. The Fraud and Ethics Hotline is on track to receive approximately 200 cases for this year as well. Yoshida reviewed IA’s appendix, including the management action item dashboard details and IA recommendation statuses, and noted that there are no outstanding overdue findings.
Committee Member De Lima noted that the Committee would be taking action on the Investigation Process Review. He asked if Yoshida had an opportunity to review Committee Vice Chairperson Dwight Takeno’s recommendations. Yoshida confirmed that she reviewed the memorandum from Committee Vice Chairperson Takeno. Committee Member De Lima asked if Committee Vice Chairperson Takeno’s recommendations would require an allocation of hours. Yoshida confirmed that IA would need to allocate hours.
Committee Member De Lima asked how the recommendations would change the audit plan overall. He stated that the Committee needs to discuss the ramifications because the report suggests that the Committee postpone action until revisions to the decision-making manual are ascertained. He noted that when IA presented the Investigation Process Review, the Committee was not satisfied with the content of the report, which is why it made recommendations and suggested further action. Committee Member De Lima asked if the notes and files of the former audit specialist who is no longer with the Department and originally worked on the report are still available for when IA assigns the review to someone else. Yoshida confirmed that these notes and files are still available.
Committee Member De Lima expressed concern that IA investigated 26 different school districts and analyzed protocols within each school district but did not provide the Committee with any information or suggestions as to its recommendations. He stated that IA’s summary findings were not well thought out. Committee Member De Lima suggested that whoever IA reassigns to the report should review the former audit specialist’s files to see whether there is any information regarding best practices that the Committee and Department should consider.
Committee Member De Lima asked how many hours IA would need to allocate to complete its tasks. He noted that the Committee needs to determine whether it needs to give this review priority over other IA priorities. He stated that the Committee could only make this determination once it understands how many hours the task would take for IA to complete and what the tradeoff would be.
Committee Vice Chairperson Takeno commented that he had discussions with IA regarding IA considering having a third party conduct the investigation. If this is the direction IA takes, it would not impact additional hours and current staff.
Committee Member De Lima asked where the funding would come from if the investigation was conducted by an outside agency and asked if IA has funding in its budget to pay for this. Yoshida stated that she needs to review and vet Committee Vice Chairperson Takeno’s memorandum with IA’s staff to determine what staff can and cannot complete and how many hours IA would need to allocate. She stated that she could come back and let the Committee know this information at a later date.
Committee Vice Chairperson Takeno noted that the Committee is discussing IA’s Investigation Process Review, not the Post-Investigation Decision Making Process Audit, and further noted that the idea is to continue to keep the two separate.
V. Recommendation for Action
A. Committee Action on the Department of Education’s Student Information System Post-Implementation Review
Yi Chen, Audit Specialist, IA, reviewed the Student Information System Post-Implementation Review. She stated that the Department launched the student information system (“SIS”) for School Year 2016-2017. The scope of IA’s review focused on evaluating the implementation of the SIS and confirming that business processes are working as intended. Chen stated that the scope of IA’s review specifically focused on an overall assessment of the following sub-categories that IA deemed as high risk in its project-level assessment: general implementation, access controls, input/edit interfaces, data processing and data integrity, and reports and outputs. The first objective of the review was to evaluate the SIS implementation, confirm that business processes are working as intended, and ensure that the systems are aligned with business and project goals. The second objective of the review was to review, evaluate, and test the design of procedures and controls over input, processing, and output of the SIS. The third objective of the review was to ensure that information captured is complete and accurate. The fourth and final objective of the review was to ensure that information generated is accurate, reliable, and timely. Chen stated that based upon IA’s review, IA found that the Department’s controls related to the post-implementation of the SIS are functioning at an “acceptable” level. Chen noted that an acceptable rating indicates that no significant deficiencies exist, controls are considered adequate, and findings are not significant to the overall unit.
Chen stated that IA made two main observations in its report. The first is a lack of data access controls for transferred employees, and the second is a need for increased training and support provided to the field. IA rated its first observation as “moderate.” Chen explained that access to the SIS is based on Department authentication. Only Department employees are registered on the Department’s active directory. Based on discussions with management and the test work performed, IA noted that when users transfer schools, their access is removed upon notification. Chen stated that staff has to rely on schools to communicate the separation. However, the Information Technology Information and Support Section performs access checks throughout the year to monitor proper access. She stated that multiple checks are performed, including checks on users that do not access the system and users with no school or “tool right” attached. Chen noted that the Information Technology Information and Support Section will disable the account if the account has not been accessed or is no longer in use.
Chen stated that based on testing performed for access removal for transferred or separated employees, IA noted seven out of 65 users with active access where the user transferred to a different school but still had access to the prior school. In addition, IA noted one out of 28 users with active access where the user transferred to a different office but still had access to the super user account. Chen noted that management confirmed that all user access noted in the observation has been corrected to reflect the proper location and that the employee with super user access had their access removed. Chen stated that the impact of this observation is that lack of data access controls for transferred employees may possibly lead to access to confidential records by unauthorized users. IA recommends that management should identify methods to automate processes to disable transferred employees’ access to the SIS. Chen stated that the Office of Information Technology Services (“OITS”) will investigate methods to address the provisioning gap where current manual processes cause delays or inaction to occur with regards to transferred employees.
IA also recommends that reminders should be sent out to the field to submit a request to disable access when transferred employees no longer need access to the SIS. Communication to the field should also be part of training. Chen stated that management currently sends reminders to the field as part of its year-end deadlines and procedures for the SIS. OITS sends communication to users via the SIS messenger feature, posts on the intranet, and posts via the Lotus Notes Memos & Notices database. OITS mentions these procedures during training sessions. Chen stated that OITS will review the communication and training content to determine if the messaging needs to be clarified or emphasized.
Chen reviewed IA’s second observation and stated that IA rated this observation as “low.” She explained that IA conducted a survey to all employees classified as “Registrar.” The purpose of the survey was to get a sense of the field’s feedback regarding the implementation of the SIS. IA sent out 87 surveys and received 64 completed surveys. The survey participants expressed a number of positive comments as well issues and concerns that their schools encountered during the implementation of the SIS. Chen stated that IA noted that 22 out of 64 registrars responded about the implementation of the system and expressed that the Department should pilot the system prior to a statewide rollout, the Department should start earlier transitions, the Department rushed the implementation, and some schools had difficulties with data transitions. IA noted that 48 out of 64 registrars responded about training they received for the system and commented that there needs to be increased training. IA also noted that 33 out of 64 registrars responded about the support that they received from resource teachers and commented on the importance of resource teachers.
Chen stated that the impact of the second observation is that the lack of addressing the need for increased training and support provided to the field may possibly lead to inefficiencies with learning a new system, inconsistent application of procedures, and confusion and frustration for the field.
Chen stated that IA recommends that management should consider piloting a system prior to a complete rollout for any new system implementation. She detailed that the SIS was done as a complete rollout due to the unsupported, end-of-life state of the legacy system with eSIS. The vendor was no longer supporting the product and the Department was the last district on eSIS. In addition, student-centered systems can be a challenge to pilot or rollout in phases due to the mobility of the students and their ability to move from school to school or complex to complex. A pilot or phased approach in such cases may cause additional burden on the schools to have to use two different systems until the full deployment is completed. Chen stated that OITS is already changing its approach to technology deployment and starting with pilots where it is feasible to do so.
Chen stated that IA also recommends that management should provide more training to the field. Training should be continuous throughout the school year and cover different features in the SIS. Management should offer training continuously to different user groups, and management could consider allowing teachers to earn professional development credits through training sessions. She noted that OITS trainers will continue in-person training on critical operational processes of the SIS and will incorporate available technology solutions to supplement in-person sessions with online training, webinars, and knowledge-based resources. Chen highlighted that upon initial system implementation, the Department identified and trained a teacher cadre to provide additional training resources via a train-the-trainer model. However, this model was not sustainable long term as a result of staff transfers to other positions or separation from service.
Chen stated that IA’s last recommendation is that management should review the effectiveness of the support provided to the field to determine if resources are efficiently allocated and staffing is adequate to meet current needs and future project goals. She noted that OITS will review current staffing responsibilities and workload to determine where efficiencies can be implemented. Based on requests from users for the implementation of additional features and modules, OITS anticipates the need for additional staff. As part of its biennium budget request, OITS has requested additional staff to assist with training and six additional full-time employees.
Committee Chairperson Catherine Payne commented that issues that IA’s raised in its report seem to have been embedded in management’s response.
Committee Vice Chairperson Takeno asked if the Department’s training for the SIS rollout was mandatory. Chen confirmed that the Department’s training was mandatory for all employees. Committee Vice Chairperson Takeno asked if the Department had a system to record who attended the training and who did not. Chen stated that there were records that showed who attended and who did not attend. Committee Vice Chairperson Takeno asked if there were assurances that those who were unable to attend were told about needing to attend. Chen confirmed that there were. Committee Vice Chairperson Takeno asked if the Department provided surveys to attendees after the training was completed to gather feedback. David Brookshire (“Brook”) Conner, Assistant Superintendent and Chief Information Officer, OITS, stated that the Department conducted surveys after training was completed.
Committee Member De Lima commented that one of IA’s observations is that there is a need for continuous training. He asked if IA ascertained how the Department conducted training and asked whether employees are able to access training through video. Chen confirmed that training is conducted through video, in-person, and through the SIS website. She stated that training can be viewed at any time.
Committee Member De Lima stated that one of the components of the SIS includes the parent portal through which parents are able to view attendance, grades, class schedules, and assignments. He asked if IA surveyed parents to ascertain their level of comfort with the SIS. Chen stated that IA did not survey parents. Committee Member De Lima asked why IA did not survey parents. Chen explained that surveying parents was out of IA’s scope of its review. Committee Member De Lima asked why surveying parents is out of IA’s scope. He stated that the scope of the review is to ascertain the implementation of the SIS to confirm that it is working as intended, which includes determining whether parents are able to engage through the parent portal.
Committee Member De Lima stated that one of the criticisms and focuses of the Board and Department’s Joint Strategic Plan is community and parent engagement. He noted that the Department invested resources into this system and emphasized that he has been vocal regarding the need to ensure that parents can make use of the SIS so that they do not need to fill out index cards and forms each school year. He stated that the phase one of the Department’s implementation is near completion of which the parent portal is part. Committee Member De Lima commented that one of the critiques of IA’s Investigation Process Review was that IA did not engage stakeholders, including unions that are part of the investigative process, to gather stakeholders’ feedbacks regarding the Department’s investigative process. He stated that parents are a group of interested participants in regards to the SIS and stated that he is trying to ascertain why IA did not survey parents. Committee Member De Lima stated that he understands IA’s scope, but it does not make sense to him unless the parent portal is not part of the SIS. He asked whether or not the parent portal is part of the SIS.
Conner explained that the Department provides schools with flexibility and allows schools to choose how to engage parents. He stated that it is not mandatory for all schools to use the parent portal. Conner noted that only a small number of schools use the parent portal, and one of the areas that the Department is reviewing is how to encourage the use of the parent portal, specifically how to tie the parent portal to online enrollment to alleviate the forms and index cards to which Committee Member De Lima alluded.
Committee Member De Lima stated that Conner’s response creates more questions. He asked why more schools are not engaging with the parent portal and asked why schools are experiencing hesitation. Committee Member De Lima asked why more parents are not clamoring for the parent portal. He stated that he knows parents who have used the parent portal and noted that parents with students in private school have access to a parent portal. He stated that if only a small number of schools are using the parent portal, it is important to engage parents to determine why some schools are using the parent portal and why other schools are not. Committee Member De Lima stated that IA highlights important observations, but it needs to vet issues if it is spending resources to get to the root of the problem. He emphasized the importance of parent and community engagement and stated that if IA is analyzing parts of a system where parents are involved, it needs to completely vet out issues.
Conner expressed appreciation for Committee Member De Lima’s feedback and stated that these are legitimate concerns. He stated that the Department will take these concerns into account as it rolls out more online capabilities to engage parents. Committee Member De Lima commented on special education components and the need to reprioritize. He suggested that he meet with the Department to discuss these issues in more detail. Conner stated that he would be happy to meet.
Committee Member De Lima stated that he does not agree with approving IA’s review and needs to review it further. Committee Chairperson Payne suggested approving IA’s review to include Committee Member De Lima’s concerns. She stated that the Committee could note that IA follow up on the specific issue of parent engagement and conduct an assessment of schools using the parent portal. Committee Member De Lima suggested that the Committee receive and file IA’s review, which does not mean that the Committee approves nor disapproves the review. Afterwards, it can engage in further dialogue with the Department and consider whether it wants IA to follow up on the Committee’s concerns or include this follow-up in phase two of implementation. The Committee can determine whether it should take further action dependent on subsequent proposals.
Committee Member De Lima moved that the Committee receive and file the review. Committee Vice Chairperson Takeno seconded.
Yoshida clarified that when IA first conducted its risk assessment for its internal audit plan, this project was included in its audit plan. When IA begins its audit for this project, it needs to conduct another risk assessment for this area because it cannot audit everything for every area. She explained that areas with high-risk ratings during the project-level risk assessment is how IA determined the scope for its Student Information System Post-Implementation Review. She stated that IA can create another project and provide the Committee with another report if the Committee wants IA to audit another area.
Committee Chairperson Payne asked if this audit covered the scope that it was designed to cover. Yoshida confirmed that it did. Committee Chairperson Payne stated that she would like more information on what would entail an addition of another project. Yoshida explained that IA could add another project. She detailed that IA conducts reassessments of risk assessments and interviews management and Board members every year. She stated that the Committee is welcome to participate in these interviews to express areas of concern so that IA can include these areas in its audit plan as well as within individual projects, including areas that the Committee wants IA to address or review.
Committee Member De Lima reviewed the objectives of IA’s review and stated that one of the objectives is to ensure that the SIS is working. He stated that IA defines the components of the SIS in its report and reviews the implementation of these components, including enrollment, roster, attendance, grade book, registration, demographics, report cards, instruction report, and parent portal. He stated that IA makes determinations regarding whether data is accurate and protected and whether the system is working. He stated that the parent portal relates to IA’s second observation. He stated that parent and community engagement are major components of the SIS, and IA made the determination that parents are satisfied even though the parent portal is not being used by all schools and IA and the Department do not know why. He stated that he is not satisfied with the effort put into the analysis of Phase I. He stated that IA’s review is acceptable in terms of its scope, but IA did not review the parent portal or consider that component important.
Committee Chairperson Payne stated that the parent portal was not included in IA’s scope and objectives. She asked if Committee Member De Lima is suggesting that IA should have included the parent portal in its scope and objectives. Committee Member De Lima reviewed IA’s objectives and project goals and commented on the importance of including the parent portal. Committee Chairperson Payne expressed concern over Committee Member De Lima’s suggestion. She stated that the areas he brought up could provide good information, but IA may need to do another project to address and review these areas.
Christina Kishimoto, Superintendent, stated that the Department can conduct a follow-up and provide a report regarding the scope of work and forward-facing work. She stated that the parent portal will be included in continued work for the following three phases and commented on understanding why the Department rolled out the system in the way that it did. She stated that after reviewing this report, the Committee could make a determination as to whether it wants IA to review this area specifically.
Committee Member De Lima stated that if IA is going to spend resources, it needs to vet and review the different components of the SIS, including the parent portal. He stated that the Department has 180,000 students and many parents and needs to start focusing on its resourcing, surveying stakeholders, receiving feedback, and ensuring ease of use. If schools are not using the parent portal, the Department needs to advocate for implementation. Committee Member De Lima stated that if IA and the Department are working on projects and initiatives that involve stakeholders, they need to engage with stakeholders, including union representatives and parents, rather than limiting scope and resources. He emphasized the importance of involving stakeholders and considering their input regarding decisions that need to be made.
Kishimoto noted Committee Member De Lima’s concerns and emphasized the importance of the Department and IA having a way to move forward.
Committee Member De Lima withdrew his original motion to receive and file IA’s review.
ACTION: Motion to accept the Student Information System Post-Implementation Review, as described in the Department’s memorandum dated November 1, 2018, with concerns noted by the Audit Committee regarding feedback from parents on the parent portal and information on why schools are not using the parent portal (De Lima/Takeno). The motion carried unanimously with all members present voting aye.
B. Committee Action on recommendations concerning Department of Education’s Investigation Process Review
Committee Vice Chairperson Takeno stated that on September 6, 2018, the Committee referred IA’s Investigation Process Review to the Human Resources Committee with a charge to conduct a further review and assessment of IA’s report. The Human Resources Committee was instructed to return the matter back to the Committee with a report of its review. Human Resources Committee Chairperson De Lima delegated this responsibility to the Human Resources Committee Vice Chairperson to look into the matter and provide a report to the Committee with recommendations. He stated that the Human Resources Committee Vice Chairperson met with representatives from IA and the Office of Talent Management to seek additional clarification and insight and analyzed and considered comments made by Committee Members at the September 6, 2018 meeting.
Committee Vice Chairperson Takeno stated that based on his review, he recommends that the Committee direct IA to continue its work with Department management representatives or designees to revise the Investigation Process Review, as necessary, to supplement the Investigation Process Review with additional facts, evidence, and documentation in support of Observation Number 1, “[i]nvestigations are not always free, both in fact and appearance, from potential impairments to independence”; include a description of the current process that explains how a complainant can appeal a decision or bring an issue to a higher level in the organization if the person feels their complaint was mishandled; clarify whether the existing reporting structure allows for independent review and whether risk can be sufficiently mitigated using existing structures, in particular the Department’s Fraud and Ethics Hotline; and clarify whether there are specific system-wide policies, procedures, protocols, and/or guidelines to determine whether to investigate internally or refer or escalate a complaint to higher authority or agencies and how consistently these processes are followed and used by principals across the state. After the completion of this, IA is directed to transmit the revised review back to the Committee for consideration, either as the revised Investigation Process Review or integrated into the pending Post-Investigation Decision-Making Process Review.
ACTION: Motion to accept the recommendations concerning the Investigation Process Review as described in Human Resources Committee Vice Chairperson Dwight Takeno’s memorandum dated November 1, 2018 (Takeno/De Lima). The motion carried unanimously with all members present voting aye.
Committee Chairperson Payne adjourned the meeting at 12:15 p.m.