Measuring Error, the Burger King Approach
Since I'm mostly one deep in the data analysis department, I'm always looking for a way to make my life easier. It's taken a bit; but we've come up with a standard format for parsing data in an Excel workbook. MatLab and the HAL 9000 unit are well beyond my current capability; so that's Lenny's department. But we do need a socially acceptable method of measuring performance and analyzing error so that we share our progress or lack thereof. As I noted in my previous blog, there are statistics, damn statistics and lies. So, I figured I'd compute mean, median, standard deviation, standard error and root mean squared (RMS) error and publish those. Folks can take a Burger King approach and have the data their way. Objectively, we want good accuracy even when maneuvering under G (or in turbulent, gusty conditions) and subjectively we want sufficient damping to make the otherwise "noisy" AOA signal usable for the stick monkey push/pull matrix. The new standard error table provides objective data. The subjective piece is a bit more difficult; but our primary technique for sharing that with folks that haven't flown with the system is the use of video.
In simple terms, here's what the Burger King menu provides: The mean is simply the overall average of the data. The median is the data point that occurs at the mid point of the data set. The standard deviation is a measure of how spread out the data are, less deviation equals more homogeneity. 68% of all of the data fall within a standard deviation of the mean (i.e., the meat of the bell curve). Standard error is a measure of the statistical accuracy of an estimate, equal to the standard deviation of the theoretical distribution of a large population of such estimates. Suffice to say, the smaller the standard error, the better. RMS error is a measure of how far the data points are from the regression line, which is just a fancy way way of saying how concetrated the data are around a line of best fit. Like standard error, smaller is better. RMS error has the tactical advantage of capturing proportionality of the error. As I discussed in a previous blog, it's best to look at absolute error so sign convention doesn't deliver an artificial warm fuzzy that the error is smaller than it actually is. For example, if you average an error of -1 deg and 1 deg, you get an average error of 0, when, in fact, the actual error averages 1 deg.
Figure 1 shows the new standard error table we are using for analysis. Figure 2 is the plot of the data (1G, power off stall). Raw delta from the corrected boom angle is shown in the red portion of the table, and absolute error is shown in the green portion. The blue line in Figure 2 is recorded V3 alpha, which corresponds to the Jan column in the green portion of the table under "Pitch Curve Error."
Dynamic Boom Correction
In a recent blog, I was lamenting about the accuracy of the Dynon EFIS we are using as one of our pitch references. Another challenge has been developing an effective dynamic correction algorithm for the air data boom on our test airplane. At 1G during stable flight, we need only adjust the boom alpha for installation error, incidence and upwash; but when we start to maneuver we have to also adjust for G (boom droop) and pitch rate, accounting for the location of the vane relative to the CG of the airplane. Our boom correction algorithm starts with raw boom corrected for upwash (Figure 3). Upwash was measured during stable, 1G trim shots and is a linear regression of raw boom data compared to known pitch, both adjusted to the fuselage reference line to compensate for installation error of the boom and pitch source. We then adjust alpha for boom installation error (+0.3 deg) and to the chord line of the wing (+0.5 deg) so that we are effectively measuring geometric angle of attack (corrected for upwash) with the boom. The V3 computes geometric angle of attack, so this gives us an apples/apples comparison.
The next element in the boom correction algorithm is deflection due to G load. It makes sense that the boom has mass and is going to bend down under G. To determine how much, we added weight at the boom CG and determined that the boom deflects .1264 degrees per G. Since our static condition is 1G, the formula for this element is -(.1264 x (G-1)).
The last element in the correction is pitch rate compensation. The alpha vane of the boom is 24" ahead of the leading edge of the wing, which works out to 3.08 feet ahead of the center of gravity at test weight. To determine pitching velocity correction, we multiply distance from CG by pitch rate/TAS [FPS]. Pitch rate is measured by the V3 IRU. The formula for this element is -(3.08 x pitch rate in deg per second/true airspeed in feet per second).
There is also an aerodynamic correction factor; but since we are operating below .3 Mach, we are considering that to be negligible. Thus the final formula applied in Excel is: =(1.0734*[measured AOA]+2.5028)+0.3+0.5-(0.1264*([measured G]-1))-(3.08*([measured pitch rate]/[TAS in FPS])).
Wind Shear, Revisited with Data
Back in our June blog, I detailed some wind shear I encountered at the end of a test sortie on 12 May. I happened to have the back-up ASI camera installed that day; so in addition to the data from the Gen 2 system and the boom there is a recording of the analog airspeed indicator.
Using the Burger King technique, I thought it would be interesting to take a look at how well the system performs during a significant emotional event like wind shear. We've been engineering for performance under gust loads; so a wind shear event is a perfect natural laboratory.
I re-edited the video I previously posted and uploaded it to YouTube in high resolution. You can view it here. Figures 4 and 5 correspond to the start of the video, 30 seconds prior to the actual wind shear event, the most significant of which occurs just prior to landing. The difference between the two charts is the smoothing applied to the AOA computation in Figure 4. We've subsequently changed our methodology for this to improve lag and are continuing to work on getting this dialed in to our satisfaction. As you can see just looking at the AOA, it was a bumpy ride down final. This is emphasized when you look at the standby airspeed indicator in the video, which is dancing around for the duration!
Figures 6 and 7 show the final 15" prior to the bottom dropping out. Note the difference between the boom and the V3 computed alpha. Generally, the low inertia vane leads the high inertia wing/airplane; but as soon as the actual event occurs, roles reverse and the pressure solution "out peaks" the boom. Frankly, we don't fully understand the dynamics yet; but figure that a picture is worth 1000 words. Figure 7 shows the same time slice, but uses instantaneously computed V3 alpha.
Figure 8 show the actual severe wind shear event itself (with "severe" wind shear defined as a wind change greater than 15 knots). If we average the EFIS and V3 computed indicated airspeed, the wind shears 15.395 knots in 1.3 seconds during the landing transition (likely an effect the the tree canyon that surrounds my home runway). The NACA 23013.5 airfoil on the mighty RV-4 stalls at 20 degrees geometric angle of attack (see Figure 2 above). Compare that to the V3 computed AOA at the peak of the grey curve and yellow spike. It was truly a "glove save" event. The boom measured a peak alpha of 17.24 degrees; but as I stated earlier, we don't fully understand these dynamics yet, and as the video shows, the airplane was definitely stalled during the event. It was damn fortunate the wheels where close to the turf when this occurred. The audio stall warning is also evident in the video.
All's well that ends well; but in hindsight adding full power and initiating a go-around would have been a smarter option under these circumstances. Being in the proper attitude was the thing that saved my bacon when the wheels touched; and if it's not evident, I'm working hard to fly the airplane throughout the landing roll to get safely to a stop. This is a classic example of a (barely) salvaged unstable landing. Thanks again to Van for engineering one of the best airplanes I've had the pleasure to fly: lots of maneuverability, power and aerodynamic margin to compensate for a momentary lapse of reason (shout out to Pink Floyd).
Chime In
There are lots of smart folks reading this blog. The reason that we're working in a fish bowl is to make sure that our methodology is out there for the world to see and comment on. If you know more about this that us (you know who you are!); don't hesitate to comment...especially if you spot an error! Please drop a line or post over on the forum. We are doing our level best to correctly apply the scientific method, and can always use help, critique and correction. Please don't hesitate to chime into the conversation.