Nothing is over until the paperwork is done. We are fans of “just enough, just in time analysis…” i.e., just enough post-flight analysis to glean what we need to move on to the next sortie. We also make lots of mistakes, because we take big steps; and the learning curve is always steep every time we tackle a new challenge. Everyone on our team brings something to the fight; but most of us would rather fly, build, test, write code, drink beer, sweep the hangar, shove sharp sticks into our eyes…pretty much ANYTHING other than sitting in front of a computer doing analysis. However, if this is going to be a worthy effort, we have to be able to quantify performance unambiguously; and at a 50Hz recording rate, we don’t have any shortage of data to look at. I'd love to say "we're there," but this analysis is still a work in progress. However, I thought I'd share what we know so far.
1G Performance
A 1G condition constitutes normal flight for most folks, most of the time. So, step one in the process is to validate performance under “normal” flight conditions. We have to accurately capture the entire “alpha vs coefficient of pressure curve” so we can measure actual angle of attack (either absolute or geometric) throughout the entire speed range of the airplane: stall to Vmax. Since we are shamelessly building on the work started by Dr. Dave Rogers and his team; we adopted their design requirement for 1/4 - 1/2 degree AOA accuracy. So, the very first parameter to assess is how well the V3 box is calculating alpha under normal flight conditions and are we meeting the design objectives of accurately capturing the curve and measuring alpha within 1/4 to 1/2 degree overall.
Generating Data. 1G data is gathered using conventional trim shot flight test techniques. We’ve standardized on a stable, six second trim shot per leg. This gives us 300 data points per shot. "Trim shot" is just a fancy way of referring to a stable GPS speed run. For this to work, the stick monkey has to stabilize on parameters of airspeed +/- 1/2 MPH IAS, and pitch +/- 1/2 degree and altitude +/- 100'. This requires smooth air, so I generally takeoff just prior to sunrise to be able to complete an hour or two of test before the solar-powered cumulo-bumpies start. Each set of runs is flown at a fixed power setting, with each leg at an identical IAS. After some complaining by yours truly, Lenny added a “data mark” function to the software; so once I’m on parameters, all I have to do is press the volume button with a long push, and then it’s easy to find what I’m looking for in the ginormous data file post flight. And lest I sound like I’m implying I have a set of golden hands that can fly uber tight parameters; I’m not. This takes lots of time, do-overs, smooth air and 100LL. It’s the aviation equivalent of painting the white rocks red in time for the next inspection. For anyone that has reached the end of the internet, tired of watching paint dry, waiting for water to boil or otherwise run out of things to do, all of my tapes are up on you tube. I’m willing to share. The tapes serve two purposes: first, I can voice record parameters and catch them in de-brief so I don’t have to write down everything in flight; and second the other folks on the team can share my misery. It’s also damn handy when something anomalous occurs.
Math in Public. To quantify things, we need to establish some sort of “ground truth” reference to compare the V3 solution to and pick an appropriate quantitative analysis technique to generate some error numbers. For ground truth, we can either use pitch (corrected for any installation error) or the air data boom (corrected for upwash and installation error). It is generally accepted to use pitch for this analysis since q ± g = a where q is pitch, g is flight path angle and a is true angle of attack...Sorry purists, my web publishing software doesn't accommodate greek letters. In proper engineering circles, that would be theta and gamma; but we'll have to settle for q and g. In a level trim shot, flight path angle is zero, thus q = a. In the RV-4, the DY-10A EFIS has an installation error of +0.3 degrees relative to the fuselage reference line (FRL), so a correction factor of -0.3 degrees is applied to recorded pitch to derive true angle of attack during a stable trim shot. To make sure we’re apples/apples in our analysis; the V3 is now computing geometric angle of attack, which is the difference between the relative wind and the chord line of the wing. The difference between the FRL and the chord line is the angle of incidence. In the RV-4, incidence is fixed at +0.5 degrees. Thus, to adjust measured pitch we first correct -0.3 degrees, then adjust +0.5 degrees. Mathematically inclined types will just cut to the chase and add +0.2 degrees to recorded pitch to figure out where the chord line of the wing is pointed…
The next step in the analysis is to figure out what statistical method to use to quantify performance. While it may seem that using a simple average and standard deviation will work; that actually presents a false picture, since some errors can be negative and others positive in the same data set. These negative/positive errors tend to cancel each other out, thus giving a non-statistically correct rosy picture of how well the system is working. Thus, it makes more sense to look at absolute mean error where we average the absolute value of each error. This technique makes sure that we are capturing the true magnitude of the error. After we’ve computed absolute mean error, we can then look at a standard deviation and get a pretty good warm fuzzy how well we are capturing the alpha curve. Another appropriate statistical technique is to compare V3 computed alpha to expected alpha (pitch corrected to chord line) and compute root mean square error. This technique has the advantage of capturing proportionality. For example, at 1 degree AOA, a 0.3 degree error is more significant than a 0.3 degree error at 12 degrees AOA. Since there are three types of numbers used in analysis: lies, damn lies and statistics; I’ll use both techniques in the tabular data and let folks decide for themselves where things fall on the “lie-statistic” continuum.
Now here's the rub: we don't know how accurate the Dynon EFIS pitch reference in the RV-4 is. I know it's pretty good, I've been flying IFR behind it for a long time; but Dynon doesn't publish accuracy and the FAA TSO requires pitch performance +/- 2 1/2 degrees--not exactly a warm-fuzzy number. Obviously that TSO was written when gyros were mechanical and powered by vacuum pumps; but it leaves us with a less than satisfying "truth" source for analysis. At least for now. I am, however, comfortable posting this as a nominal "first look" as the results are consistent from sortie to sortie. Our next step is to see what's in the art of the doable as our IMU work matures. Barring progress on that front, we'll have to bolt some more reference instruments into the RV-4. The next step is to fly the new IMU software in some representative trim shots to cross-check performance of that chip against the basic EFIS and boom performance.
Tabular Data
Overall 1G results are shown in the table in Figure 1. Three sets of dedicated trim shots were flown (plus a bonus flaps 40 set of runs) in January, June and August 2020. Results for each sortie are depicted, and then each line is averaged horizontally to produce an "overall average" in the right hand column.
Analysis Workbooks
The error analysis workbook that was used to prepare the table in Figure 1 can be downloaded here. Note that each data set includes an upwash plot as well as a “pitch-derived geometric” curve—that’s the algorithm that the V3 is using to compute geometric alpha. Lenny is currently working on taming the on-board inertial measurement unit; and when that’s complete, the system will use pitch measured by the IMU for calibration. Our plan is to use that capability and the WiFi interface to create a “calibration wizard’ that will simplify and step the pilot through the calibration process--a calibration that results in consistently accurate results, regardless of the airplane it's installed in. Integrating the IMU we'll be a topic of a future blog--talk about a steep learning curve! We are currently learning how to invent an EFIS on a ten-dollar platform...We'll recompute overall accuracy and publish updated workbooks after we've done more IMU test flying.
This workbook incorporates the physics the heart of our project and is based on Dr. Roger’s earlier work. What’s important is the ability to put GPS run data in the left side and spit out absolute alpha calculations on the right. Our concern with the accuracy of the Dynon pitch reference doesn't apply to this physics-based approach to calculate absolute alpha. Recall “absolute alpha” is the difference between the relative wind and the zero-lift line. We made the shift from absolute alpha to geometric alpha for two reasons. The first is that “geometric” alpha is what we are all taught in pilot training; but the primary reason is that we can derive geometric alpha directly from pitch using the q ± g = a equation and the capability of the IMU to measure pitch and flight path angle. A couple of things to keep in mind as you look through this. First, it’s necessary to compute an “aircraft lift curve slope” (which is a function of the airfoil section) and input that value along with other aircraft parameters on the upper left portion of each tab. Also, I use a spreadsheet developed by Kevin Horton to derive TAS that is manually input in the table in the upper left portion. It’s necessary to use horsepower as part of the aerodynamic calculations (since thrust required serves as a surrogate for drag). The algorithm in the spreadsheet is derived from Lycoming Curve 13381 and is only applicable to the 160 HP O-320-D2J engine in my RV-4. The low drag characteristics of the airplane and the fixed-pitch Catto prop make HP calculations at low power a bit of a challenge, and it’s also difficult to measure alpha at higher AOA after flow separation begins; thus we have a couple of techniques we use to “QC” (quality control) the validity of a data point. The first is to plot coefficient of drag vs coefficient of lift squared. The second is to plot thrust horsepower required times velocity vs velocity to the 4th power. In each case, these plots should form a roughly straight line that mirror the lift curve. If any points significantly diverge from the line, the quality of those points is considered suspect. The primary outputs of this workbook are shown in Figures 2-4.
Feedback. We ALWAYS welcome feedback! There is lots of engineering and flying talent in the EAB community. I'm publishing our homework and analysis tools so that anyone can cross-check our thinking and our math. Don't hesitate to get in touch or post over on the forum if you spot errors, have suggestions, questions or want to participate in the discussion. The more collaboration, the better.