Examination of Witnesses (Questions 19640
- 19659)
19640. What has been available to you, you have
looked at?
(Dr Hunt) Yes.
19641. With that in mind, can you take us through
the sources of uncertainty with the Findwave model in mind, please,
starting with model assumptions?
(Dr Hunt) Yes, in general I have been interested
in trying to understand why it is that predictive models have
uncertainty built into them. It is then a question of how can
we make them better, and if we understand the sources of uncertainty
then we can move towards improving those models. I have divided
the areas of uncertainty into seven areas which you see listed
before you. I can go through them very quickly one by one. At
some stage or another, before you begin to write software, to
make predictions you have to decide which laws of physics you
are going to use. There are various things we need to do.[18]
For instance, we need to understand that elasticity is a linear
phenomenon, by which I mean that if you push on something with
twice the force you will get twice the deflection. Interestingly,
some of the simplest problems are not linear. The example I like
to use most often is that of a tuning fork, which, once hit, you
cannot hear until you place it onto a table, and when you do place
it onto a table it is much, much louder than it is in free air.
The only way you can explain that is using a non-linear model.
I am not saying this is at all relevant to the exact process of
modelling vibrations on railways, but it is an example whereby
a non-linearity is present and if you inadvertently ignore it
that can lead to a significant error. Damping models are very
important. Damping is all about how quickly things die away. If
I hit a piece of metal, it rings like a bell; if I do the same
to a similar sized piece of rubber, it is dead. Rubber is a highly-damped
material and a piece of metal is a very low-damped material. In
a construction of a tunnel in soil, we have a mixture of all sorts
of materials and all sorts of different damping characteristics.
Understanding the damping in detail is essential to get an accurate
model. As a body of scientists and engineers, we do not understand
damping. The next slide refers to the excitation model. We need
to know about the way in which the wheel which is not perfectly
round moves along the rail which is not perfectly smooth. The
models that we have which are presented in many conferences and
many journals go a long way to understanding the excitation model
but they are certainly incomplete and there are errors which arise
from that. Next Slide please.[19]
All computer programmes are inherently inaccurate because you
have to make some kind of assumption about the real world to fit
them into computerised language. For example, if I wanted to find
a circle, you have to do what Archimedes did when he was trying
to get a value for pi: you divide your circle into perhaps
six points to make a hexagon or eight points to make an octagon.
How many points do you need around a circle before you can really
call it a circle? How many points do you need around that circle
before, for instance, the value of pi would be correct?
That is an example of what I have called here the discretized
models needing to be checked for convergence. Also, we know by
using any software that we do find bugs and those bugs need to
be found out and sometimes they can be hard to find. Also we sometimes
find that when we want to run a computer model quickly there is
a limit to the amount of time and resources that we have to cut
our model smaller or so on and that leads to errors. That is what
I have called the truncation error. The slide headed "Shoe-horning"
the picture on the left shows a train running through a tunnel
and the tunnel is not perfectly circular and the ground around
is not perfectly flat.[20]
There are roots from trees and there are inclined layers and perhaps
there is a river nearby, yet the computer model will not have
all these things and I have to use my judgment to fit the things
that I am trying to model into the model. What kind of assumptions,
what kind of judgment might I use to deal with these things. Different
people will make different assumptions and therefore they will
get different answers. My guess is that two different people using
Findwave, for example, or my own Pipe-in-Pipe model will get different
answers, even given the same measured data for soil parameters
and so on. Next slide please.[21]
The next area is data gathering. We know we have to try to figure
out what the soil properties are, what the tunnel characteristics
are, what the foundation characteristics are, where the water
table is, the geometry of the buildings and so on and so forth.
We have to know how accurate that data is because, as the adage
goes, "garbage in garbage out". I am not saying we are
putting garbage in but we are putting data in which has some sort
of error associated with it and, inevitably, if there is uncertainty
in the data provided to the model, there must be uncertainty coming
out. We also assume, for instance, that if we measure properties
at the ground in one place they will be the same as they might
be were they measured somewhere else. Perhaps, also, if we measure
them on the Monday they will be the same as if they were measured
on a Tuesday, or if they were measured in the summer they will
be the same as they would be in the winter. There are some things
we simply cannot measure. What are the soil properties directly
underneath the tunnel. It turns out that this is a very important
part of the model. It is very hard to obtain data for that kind
of place.[22]
I have mentioned the excitation model before. There are so many
things that we do not truly understand about the excitation model.
We can presume that we understand what is going on but I have
listed here a few things: unsprung masses, vehicle speed and so
on, but we find that if we take the same train running over the
same piece of track at the same speed that we will get different
measured sound levels or measured vibration levels. Why is that?
Perhaps it is because the train is not following exactly the same
track line on the rails or perhaps it is because the interaction
between the wheel on the rail is not exactly what we think it
might be. But there is some uncertainty there.[23]
The measurement point is a clear source of uncertainty. If we
take a room such as the one we are in here, we know that this
room is probably going to be different from the one next door.
What kind of detail do I need in my model to be sure that I have
got to within one or two or three or four or five decibels, a
good answer for prediction of sound levels? Likewise, whereabouts
in the room am I going to make my measurements? We all know that
if you go to a venue to listen to music, there are some seats
from which you hear better than others. The nature of rooms is
that they are variable. One thing which is shown in the colour-coded
diagram is the dB level variation you might get in the ground.
The white circle in the middle represents the tunnel. The scale
is given in metres. This represents a cross-section through the
ground 40 metres wide and 40 metres high, so 20 metres up and
down from the tunnel, and 20 metres left and right. You can imagine
that somewhere where near the arrows are pointing represents the
position where the foundations for the building might go. The
difference in colours, depending on where the arrow heads are,
go from yellow to orange-ish to light blue and dark blue. There
is a variation there of plus or minus 10dB from place to place.
This diagram shows the effect of what kind of variations you might
get with small, but reasonable within the levels of uncertainty,
changes of soil properties. You can see then, depending on exactly
where you think the foundations are for the building, that you
might find the building will act as a receiver for vibration much
more effectively in one particular case but less effectively in
another. Do we really have enough detail in the model and our
knowledge of the building to be sure?[24]
Then I turn to validation. We really need to be sure that whatever
model we use has been used often enough to demonstrate to us all
the range of predictions. We do not just want to see the best
predictions, we want to see the worst ones as well. If we are
trying to forecast the weather we need to know that sometimes
the weather forecast is really bad and sometimes the weather forecast
is really good, so that we get some feel for how much we should
trust the weather forecast. My feeling is that, generally speaking—and
it is not just on Findwave but all models of which I speak—we
are not in a position to have followed through the validation
process sufficiently to have confidence in making accurate predictions.
I know that the world of the weather forecasting is very, very
far down the track on that. You may not know that the weather
forecast that we get is an amalgam of several collaborating or
communicating communities of weather forecasters and they all
have to agree on the weather forecast for the next 24 hours and
when they do agree our weather forecast is robust. When they disagree,
we get what we call unsettled weather or uncertain forecasts anyway.
You can see this little quotation at the bottom from the world
of Microscale Meteorologists. These are the ones that look at
pollution dispersion. If you have a chimney which is spewing out
horrible stuff, we want to know that pollution levels are modelled.
I think it is very nice that they consider that models are "only
of use if their quality (fitness for purpose) has been quantified,
documented and communicated to potential users. It may not be
appropriate to talk of a valid model but only of a model that
has agreed upon regions of applicability and quantified levels
of performance" I just do not think we have those agreed
upon limits of applicability and we do not have quantified levels
of performance. I have given two examples that Findwave used in
published literature on the next slide.[25]
The one on the left was published at a conference I attended in
September 2004 and the one on the right Findwave has used against
the same set of measured data published as part of the validation
process for Crossrail. The next slide shows an enlargement of
both graphs.[26]
The red line is a prediction made by Findwave for the Crossrail
validation and the blue line is a prediction made by Findwave
for the conference in Buxton a few weeks earlier. The black dotted
line is the measured data. It is in the nature of things that
when you have the measured data in front of you it is very easy
to say, "I can see how I can make my prediction better because
I can adjust my data" and indeed my prediction would be better.
What has happened, perfectly sensibly, is that the loss factor,
which is a very important parameter which can be used to change
predictions by a large amount, has been adjusted to get better
predictions. However, the question I would ask is: How confident
are we that in our present case, when the measured data is obtained,
that the model will actually agree very well. When we have the
measured data available to us, it is very tempting to re-run the
model and to get the better agreement. In this case, you can see
the model differences where I have marked 8dB there.
19642. Chairman: Are you talking about
the model used for the whole of the Crossrail project or just
for a particular part of it?
(Dr Hunt) I am talking about models in general,
of which Findwave is one, of which my own Pipe-in-Pipe model is
another. I am talking about areas of uncertainty inherent in the
process of modelling. Therefore, yes, indeed, it does apply to
the whole project. Just to sum up then with this slide here.[27]
Let us suppose we are going to talk in quanta of 5dB because I
do not think anyone is inclined to use a finer scale, but that
plus and minus 5dB is reckoned to be really good and the best
model that we might like to think of. I feel in my own opinion
that there are six areas of uncertainties about, let us leave
out the computer model itself, I do not know whether Findwave
has been checked to make sure there is no bugs in it, and I am
not familiar enough with Findwave, but let us look at things which
are not to do with Findwave but to do with all models. I would
say it is not at all unreasonable to suppose the various assumptions
each have a 5dB uncertainty to them. How do all these add up?
It does not make sense to add them all up. You could end up with
plus or minus 25dB but we do know that models are better than
that, so this is why we have to go through a validation process.
A validation process needs to be robust, transparent and include
the best with the worst. I have no doubt that in a few years'
time as a community we will have gone through that validation
process and have a much better idea of what is a true and representative
area to apply. In the meantime, I think 5dB plus and minus is
unreasonably tight. If you look at measurements and modelling,
you will find that it is pretty much everywhere, the best you
can get is plus and minus 10dB. That is why I had this plus and
minus 10dB figure which I like to use as my figure.
19643. Mr Binley: As a complete layman
whose only concern is trying to deal with this matter, you have
opened up a whole new world to me. I want to relate the world
that you have opened up to what we are talking about with regard
to this particular issue before us. Are we talking about NC25
as a standard which is being applied and lowering it to NC20?
How does your margin of error relate to that? Is it a direct transference
and you are saying plus or minus ten points on that scale?
(Dr Hunt) Yes. I do not
think there is much point in going back to one of the slides we
saw earlier this morning, but if you imagine a slide that had
those little vertical bars, which were 5dB in length, I would
expect those bars, in my judgment, to be plus or minus 10dB which
is in fact what we see. That has been published in and around
this particular case. You will find that, indeed, the data goes
up and down, it is data which in fact Rupert Taylor has just republished
in the last few days. There is a prediction line and the data
goes up and down and around that plus or minus 8 or 10dB.
19644. It becomes pretty useless in those terms?
(Dr Hunt) It does not become
useless at all. There are standards which—I have to say
the setting of those standards and the meeting of those standards
is not my subject, I am into the modelling process—need
to be agreed upon and we need to strive for those standards and
achieving those standards is the ultimate goal. We need to know
what they are. NC25 and NC20 are standards and, likewise, octave
bands or third octave bands are agreed procedures by which we
might go towards meeting those standards. Ultimately, the question
is what do we do if we exceed those standards? Essentially, if
we exceed those standards, we are, by an objective nature, saying
this is too noisy. We are going to use predictive models to help
guide us to design a good railway so that we can in all probability
meet the standards but because there are errors and predictions
always have errors—
19645. How do we judge potential error rates
of that kind with the sorts of very, very fine figures that we
hear are necessary with regard to the process, the matter we are
discussing today presents to us? How do we relate those two things
and come to some sort of sensible judgment?
(Dr Hunt) We heard earlier
that the best studios are not built above railways so what is
happening is here we have got a railway which is being built under
the studio, so the predicted models are saying that the vibration
and sound levels will be below NC25 by a sufficient amount to
satisfy those in the recording studio; they may not be. This is
a very sensitive site and it is awkward. Ultimately, it may turn
out that it is nice and quiet and the studio can operate perfectly
normally but it may turn out that it cannot.
19646. Mr Hopkins: Your summary of uncertainties,
the possibility or probability of all of them pointing in one
direction is unlikely, it is a possibility, and, in fact, some
of them may counter each other, some may be plus and some may
be a negative, so one could take an extreme case but that might
be very rare. You have also had the opportunity of looking at
a large number of existing situations where there are tunnels
with continuously welded track, soundproofed, under London Clay
or sand or whatever, and one can look at empirical evidence from
elsewhere across the country, not just in London, to get a normal
distribution of noise effects. You are talking about the possibility
of one end of that normal distribution, an extreme end of that
noise distribution. Anything is possible. We might walk out and
be run over tomorrow, but it is an unlikely possibility, most
of us will survive. I think there are some probabilities to all
of this and it seems to me that you are saying it could be an
extreme case and therefore it might be difficult.
(Dr Hunt) I can say really
that if you look through the various published papers and what
people are presenting at conferences, what you will find on the
internet, what Mr Thornely-Taylor has published and so on, the
plus and minus 10dB looks pretty much the norm and my worry, of
course, is that people only publish their best results. If you
run a model and you get a prediction which is whatever it is and
then you have got lots of measured data and say, "I have
done something wrong here", then you say, "I am so stupid.
I forget to put the decimal point in here", so you run it
again and you get a better figure and it is good, but in the absence
of measured data, would you run the model again? I think what
you are suggesting should be done and it has not been done, and
that is to do lots a predictions, a priori predictions, before
doing measurements or at least without the measurements being
known to the people doing the predictions. That is very rarely
done, because usually what happens is the person who is writing
the paper to be presented in the ninth c International Workshop
on Railway Noise which is coming this September, those people
will have the measured data and the model predictions side by
side and will produce a nice presentation. There you will get
the plus or minus 10dB. My worry is if plus or minus 10dB is present
in the best of models, best of predictions, I think that really
is where my reason for saying plus or minus 10dB comes from. I
should also say my 10dB also comes from my own experience of running
my own predictive model which has been cross-validated with other
models, and you can very easily get plus or minus 10dB there.
You made the point earlier that sometimes these things add up
and sometimes they do not. There is a graph which I am not sure
whether it is going to be presented later on, but it shows a prediction
with a peak of 80 Hz and a couple of troughs on either side, whereas
the measurement does not seem to show those peaks and that is
entirely consistent with the fact that you get strong excitation
from around 80 Hz. It is also possible, as I showed in the previous
slide with those coloured plots, that you can get peaks and troughs
at different frequencies on either side and the two things can
cancel out or they can reinforce. It is a close-run thing. It
could be up by 10dB or it could be down which is precisely what
we see all the time.
19647. I have some knowledge of statistics as
a student and in use of my daily life, but in politics we are
used to using statistics dishonestly! Except for me because I
am always truthful but we have seen what can be done with statistics.
My point is still that even if it is this plus or minus 10dB for
each of these factors—
(Dr Hunt) I am not saying
that it is.
19648. Even if it was, the probability of them
all pointing in one direction is two times, two times, two times,
two times two, or something.
(Dr Hunt) I am not saying
that the models are out by 25dB, I am saying that plus and minus
10dB is a good working figure.
19649. My final point is are you saying that
there is no objective data anywhere about the vibrations from
trains in tunnels affecting ground buildings?
(Dr Hunt) I am saying there
is no objective statistical data of the types you are quite rightly
suggesting there should be showing predictions made over a large
number of sites by a large number of different people and collecting
the data, adding it together, and saying, "All right, this
is the mean area and these are the outliers", no, not available,
very sadly not available.
19650. Mr Newberry: Dr Hunt, what I would
like to do is on your summary of uncertainties if you are able
to indicate under the various headings which you have identified
what it is specifically about the Findwave model in terms of assumptions
and shoehorning et cetera, what it is that concerns you about
the assertion that the model can be accurate to plus or minus
5dB?[28]
(Dr Hunt) In model assumptions,
for instance underneath the railway we have a rail pad, this is
the pad which goes underneath the rail it is made of rubber. It
has got core particles in it or whatever, it might even have dimples
on it, and it is well known by the manufacturers of these pads
that these are non-linear. In fact, they are designed to be non-linear
in many cases. The manufacturers themselves cannot provide dynamic
data for these rail pads which is particularly accurate. Then
if we are talking about ballast mats or under-sleeper pads, this
is an under-sleeper pad (indicating) again this is a material
which has non-linear characteristics. I would wonder whether the
non-linearity has been included properly. Ballast itself is a
highly non-linear material. There are various models which have
been used to predict the performance of ballast. The ones I have
seen published are okay, but in my opinion I think they miss the
point. The way in which the ballast functions is different from
the way in which it supposed to function and that is a subject
for discussion.
19651. Pausing there, Dr Hunt, this is under
the subject of linearity, and you have looked at various materials
which are not linear.
(Dr Hunt) They are not linear,
and the soil itself underneath the tunnel is not linear.
19652. Is the Findwave model a linear model?
(Dr Hunt) It is not a linear
model. I know that. It has, for instance, frequency-varying damping
and frequency-varying loss factor—in fact, the very use
of the loss factor itself is a non-linear model. So it is not
a linear model, but without looking in detail at the model I cannot
say what is inside it but I would be extremely surprised if it
did include the kinds of non-linearities I am talking about because
the data that is being put into the model, as far as has been
published, does not make any reference to the non-linearities
of which I speak.
19653. Given that you are dealing with a model
that is non-linear and dealing with linear properties, what does
that lead you to conclude in terms of the accuracy, in this context,
of plus or minus 5dB?
(Dr Hunt) I suppose I should
say that one of the biggest sources of error here is in modelling
of damping, and that I notice that the Findwave model uses a frequency-varying
loss factor and that it uses the same loss factor both for compression
modulus as it does for shear modulus.
19654. What does that mean?
(Dr Hunt) That means that
waves that travel through the ground travel as pressure waves.
For instance, if I clap my hands, that wave travels through the
air as a pressure pulse but solid materials can resist shear.
So that if I were to push sideways on a block of rubber it will
resist motion, whereas if I push sideways on water and air, water
and air then these will not resist the pressure. Within soil and
within materials generally you can have two types of waves that
travel, a compression wave and a shear wave. In soils, it is pretty
well recognised that the way in which pressure waves travel through
the soil, the damping associated with it does not have a sliding
motion, any sideways motion, of the grains of soil, whereas with
shear motion there is a sliding motion. So it is perfectly reasonable
to assume that the loss factor in shear will be greater than the
loss factor in compression. I am pretty certain that that will
be the case, but in order to make those measurements and distinguish
between the two requires some pretty careful measurements, but
I know that Findwave does not distinguish between the two. It
may turn out that there is no need to distinguish between the
two, but I have not seen any evidence published or otherwise to
say that it is not important. In my view it is important to distinguish
between the two, and not distinguishing between the two will lead
to errors of, again, perhaps 5dB or more.
19655. So that is looking at soil.
(Dr Hunt) That is looking
at soil.
19656. There is an uncertainty because of those
two factors you have identified, and your perusal of the model
has not explained either at all or satisfactorily why one option
as opposed to another is being pursued.
(Dr Hunt) That is right.
It is interesting, perhaps, to say that a change of loss factor
by going from (if I get this from my memory right) 0.1 to 0.15
will give you a 6 or 9dB difference in vibration level at the
surface of the ground for a 15-metre deep tunnel. That 9dB is
already more than the 5dB that we are hoping to get from Findwave.
That change in loss factor from 0.1 to 0.15 is completely within
tolerance that is used by Findwave and which is expressed in the
data that we have had in front of us.
19657. What is the next factor?
(Dr Hunt) I do not want
to say anything about the computer model. The Shoe-horning: I
suspect that for London it is probably not too bad; we have got
fairly level horizontal layers, we have got a fairly stable water
table; the tunnel is fairly level. I say "fairly", there
are always going to be small effects—the tunnel is ever
so slightly curved. What kind of influence will that have? If
it is half a dB then we need to know that. The tunnel is not going
to be perfectly, perfectly uniform in wall thickness and the ribs
in the tunnel and detail of tunnel construction—we have
to make assumptions there. I think we have to bear in mind that
the area around Soho is full of other tunnels, other foundations
and other buildings with piles, and so forth. Do they need to
be included in the model, or do we make assumptions about the
error associated with ignoring them? The error associated with
ignoring these things, which is the shoe-horning effect, for 5dB
is a perfectly sensible and, perhaps, rather conservative estimate.
Without going through the statistical process who knows?
19658. Could I ask you, finally, just on the
question of validation, is Findwave, so far as you can see, a
properly validated model such that one can predict accuracy to
5dB?
(Dr Hunt) I do not believe
there is any model which is sufficiently validated to achieve
that kind of accuracy, so the answer is, in my opinion, no. My
own model is also not validated to that kind of accuracy.
19659. You have appeared at conferences, of
course you have, and I think you have also appeared at conferences
with the Promoters' noise witness. Is that right?
(Dr Hunt) Yes.
18 Committee Ref: A221, Model Assumptions (WESTCC-9305A-005). Back
19
Committee Ref: A221, Computer Model Correctness and Convergence
(WESTCC-9305A-006). Back
20
Committee Ref: A221, Shoe-horning (WESTCC-9305A-007). Back
21
Committee Ref: A221, Data Gathering (WESTCC-9305A-008). Back
22
Committee Ref: A221, Excitation (WESTCC-9305A-009). Back
23
Committee Ref: A221, Measurement Point (WESTCC-9305A-010). Back
24
Committee Ref: A221, Validation (WESTCC-9305A-011). Back
25
Committee Ref: A221, Findwave Validation Graphs (WESTCC-9305A-012). Back
26
Committee Ref: A221, Findwave Validation Graph (WESTCC-9305A-013). Back
27
Committee Ref: A221, Summary of Uncertainties (WESTCC-9305A-014). Back
28
Committee Ref: A221, Summary of Uncertainties (WESTCC-9305A-014). Back
|
