APPENDIX 14
Memorandum submitted by British Telecommunications
plc
ELECTRICITY SUPPLY IN NORTHERN IRELAND, CHRISTMAS
1998
This Memorandum addresses the questions posed
by the Northern Ireland Affairs Committee in respect of BT's response
to the problems experienced by NIE between Christmas and New Year
1998.
NIE HELP LINE
0345 643 643
Calls to the 0345 643 643 Help Line are routed
in the first instance to NIE's own call handling agents, based
at a number of locations across Northern Ireland. When all available
NIE agents are busy, any remaining calls overflow to the Message
Link service which provides pre-recorded information for callers.
Under normal circumstances, Message Link can be updated remotely
at will by NIE.
A day by day breakdown of the number of calls
made to NIE 0345 643 643 Help Line during the period 26 December
to 1 January 1999 is given at Annex A. The figures show that the
Boxing Day storm stimulated an unprecedented number of calls to
the NIE Help Line, which amounted in total to 651,684 calls in
the period up to 1 January 1999.
MESSAGE LINK
SERVICE
Problems were experienced by NIE with updating
the pre-recorded Message Link announcement during the evening
of 26 and the morning of 27 December. A problem was reported to
BT at 1835 hours on 26 December, though BT was, at that time,
unable to identify a specific fault. BT did recognise and acknowledge
that the NIE were experiencing problems in updating the message
remotely.
BT can now confirm that there was no fault on
the Message Link service at any time, and, following the incident,
further investigation has identified the cause of this problem.
Access to amend the recorded message on Message Link takes place
over the same lines as customer calls to Message Link. As a result
of the congestion caused by exceptionally high volumes of calls
overflowing to the Message Link service, NIE personnel were unable
to get through themselves to make the change to their announcement.
The congestion continued until the Sunday afternoon (28 December)
when the call volumes to the Message Link service started to reduce
and NIE were able to get through and change the announcement.
The Message Link service continued to work effectively during
this period. As a result of the high number of calls and complaints
that NIE received, NIE asked BTNI, on the morning of 28 December,
to mobilise some BT agents to take calls from their customers.
During the period from 28 to 31 December, BTNI call handlers dealt
with over 12,000 calls on behalf of NIE.
Reference has also been made to the fact that
some of BT's local exchanges became overloaded due to the unprecedented
number of calls to the Help Line number in such a short timescale.
BT can confirm that some congestion was experienced at a number
of our small rural exchanges on both 26 and 27 December due to
the high volume of call attempts being made to the NIE Help Line.
The telephone exchange design and route capacity are dimensioned
to cater for the normal daily busy hour traffic levels. This exceptional
surge of traffic in addition to the normal calls resulted in a
small number of our local exchanges experiencing congestion at
peak times. The most serious congestion was experienced at Lisnaskea,
Florencecourt and Claudy exchanges and resulted in our customers
receiving "equipment engaged tone" as they attempted
to dial out.
It would simply not be feasible to design the
switch or provide the route capacity to cater for such unprecedented
and infrequent surges of traffic. There would also be little advantage
in advancing the calls to the destination number if there were
insufficient lines or agents available to answer them.
BT NETWORK INTEGRITY:
CALL GAPPING
The Report comments "that BT had at times
to restrict the number of calls to NIE directly to maintain the
integrity of its network" and the reasons for this should
be clearly understood.
BT's network successfully delivered a total
of 550,133 calls (see Annex A) to the IE Help Line 0345 643 643
on the 26 and 27 December. It was able to cope with this unprecedented
volume of calls as it was a holiday weekend during which there
was little other commercial traffic calling the 0345 range of
numbers.
On Monday 28 December there was a significant
increase in the level of normal commercial telephone traffic from
post-Christmas promotions, catalogue sales and retail activity,
etc., which resulted in an overload on the national 0345 platform.
When more calls are initiated to the various 0345 numbers than
the companies employing them have the capacity to answer, BT's
network can become congested and this may prevent other calls,
including those to the emergency services, being able to get through.
When it is necessary to prevent this, and thereby to protect the
integrity of BT's network, national Call Gapping is introduced.
This means that BT limits the number of calls that are forwarded
to the dialled number at any one time. If this did not happen,
major congestion, which may impact on the remainder of BT's network,
is possible.
Between 0913 hours and 1226 hours on Monday
28 December, BT found it necessary to introduce Call Gapping because
of the serious level of route congestion that was being experienced
on the national 0345 platform. As a result of Call Gapping, the
number of calls allowed through to the NIE Help Line number was
limited during this three hour period, but overall BT's network
successfully delivered 651,684 calls to the NIE Help Line number.
IMPACT OF
POWER SUPPLY
INTERRUPTIONS ON
BT
The Committee should also be aware of the problems
that the extensive and prolonged power failures created for BT
itself in Northern Ireland during the period in question.
BT experienced a number of serious service difficulties
as a direct result of the NIE faults and power surges induced
data corruption on a number of our network control links. In total
we experienced 52 short duration exchange isolations on 26 December
which resulted in 19,000 of our customers experiencing a total
loss of telephone service for a few minutes on each occasion.
All telephone exchanges require continuity of
power supply to remain operational. Following the Boxing Day storm,
BT experienced the most extensive and prolonged power supply failures
on record. However, to minimise any disruption of service to our
customers, all BT's exchanges are equipped with standby electricity
generators, installed as part of our planned network resilience.
By design, these are only intended to cover short break interruptions
to the power supply to sustain continuity of service to our customers.
To illustrate how serious the situation was on 26 December, 80
of our telephone exchanges lost their power supply at some time
during the day. Without our standby capacity, BT would have lost
telephone service to over 150,000 customers. A number of our generators
had to run continuously for three and four days before mains power
was fully restored.
During this time BT's service was at risk. It
is clearly important that the significance of BT's services is
recognised and a sufficiently high priority assigned in any restoration
and recovery plans. Clearly these are issues which BT will discuss
further with NIE to ensure that we are given appropriate priority
in future recovery and restoration plans. Even so, it was reassuring
to note how well BT's telephone network handled such an unprecedented
volume of calls, despite the impact of such extensive and prolonged
power supply failures.
The telecommunications network infrastructure
itself also suffered extensive physical damage as a direct result
of the severity and duration of the Boxing Day storm. BT's fault
volumes increased by 300 per cent in the week following the storm:
317 poles were brought down, 1,700 realigned, 3,540 overhead lead-ins
to customers' premises had to be renewed and 210,917 metres of
aerial cable replaced.
LESSONS LEARNED
AND ACTION
TAKEN
As a direct result of this incident, BT has
already held a joint Workshop with NIE to investigate and agree
how we can work together to improve their call answering facilities.
Proposals already being discussed include:
— developing additional contingency
call answering back-up capacity which can assist NIE during periods
of heavy demand;
— the introduction of Interactive Voice
Recording (IVR) equipment into NIE's advanced services call plan
to improve the effectiveness of the call answering agents and
to reduce call handling times;
— review of the design of Message Link
access facilities to provide priority access for message updates
during periods of high call volumes; and
— recognition by NIE of the critical
importance of BT's services and assignation of higher priority
in future restoration and recovery plans.
It was reassuring for BT to see how well our
own network infrastructure stood up to the storm damage and major
power failures. It was also reassuring to note how well our own
fault reporting procedures and emergency restoration plans stood
up to the impact of this storm.
BT Northern Ireland
March 1999
ANNEX A/1
| Daily Record of Call Volumes to NIE Help Line
|
| | |
|
| | |
|
| Calls | Cumulative total
| Percentage |
| | |
|
| | |
|
| 26 December | 304,909 | 304,909
| 47 |
| 27 December | 245,224 | 550,133
| 84 |
| 28 December | 49,893 | 600,026
| 92 |
| 29 December | 31,926 | 631,952
| 97 |
| 30 December | 11,477 | 643,429
| 98 |
| 31 December | 5,294 | 648,723
| 99.5 |
| 1 January | 2,961 | 651,684
| 100 |
ANNEX A/2
| Faults Reports to 151
|
| | |
| | |
| | Normal
|
| | |
| | |
| 26 December | 1,720 | 1,024
|
| 27 December | 4,991 | 650
|
| 28 December | 5,052 | 2,201
|
| 29 December | 5,369 | 1,935
|
| 30 December | 5,184 | 1,869
|
| 1 January | 3,241 | 1,895
|
| | |
| | |
| Total | 25,557 | 9,573
|
| | |
| Fault report call volumes increased by a factor of 2.66 against normal fault levels.
|
|
