ATP - Testing and Model Validation

Section II: Testing and Model Validation

Before publishing the calculations, I wanted to make sure that the equations I was using came fairly close to real world results. This section concerns the testing methods I used to verify the calculations.

Section II-A: Verifying Velocity Calculations

After I’d created the program I set about testing to make sure reality matched up with my theoretical calculations. I fired airsoft BB's from guns with muzzle velocities ranging from 200 fps to 600 fps. I performed thorough tests measuring velocity at 5’ intervals for several different guns with the maximum range in most cases being 60’ (though in the case of the 500+ fps velocities, I tested at 10’ intervals out to a maximum range of 100’).

Also, I asked for data from other sources, people who’d collected data at different temperatures and different altitudes. I also tested the data using different size BB's (in this case, 8mm BB's and paintball BB's). For all of the data collected, I compared it to the calculations made by the program. For data collection, at each range I needed a minimum of 20 data points. Basically, the more data collected would yield a better average velocity.

When comparing theoretical velocity calculations to actual results, my goal was for the program’s calculations to be within two standard deviations of the data collected. For those who haven’t done much statistical work, a “standard deviation” is a statistical method for calculating the probability that sample data will lie within a normal distribution. For example, if I collected fifty data points for velocity and the average velocity was 243.1 fps with a standard deviation of 2.0 fps, then 68% of the data points will be within one standard deviation of the average (i.e., 68% of the measurements will be between 241.1 and 245.1 fps). Furthermore, two standard deviations encompass 95% of the data, while three standard deviations encompass 99.7% of the data. In other words, if I fired over 1,000 shots and the average was 243.1 fps with a standard deviation of 2.0 fps, 997 of the shots would have been between 237.1 and 249.1 fps.

Having said that, I found that the results I had calculated matched to within two standard deviations of the collected data, with most measurements being well within one standard deviation. Considering that the standard deviation was usually less than 2.0 fps, I would say that the calculations are very, very accurate.

Here’s an example of how I compared the theoretical calculations to collected data. I used a Chrony F-1 and a Chrony Beta Master as the test platforms, with the F-1 sitting 1-2 inches in front of the muzzle, and the Beta Master down range. Additionally, I had the gun sitting in a vice (which took some tweaking in order to line it up with the chronograph). On each of the days I took the measurements, I recorded the temperature and used it for making the calculations, with some of the measurements being taken indoors and some being taken outdoors in excellent weather (though I waited for windless days to do the outdoor work). Additionally, most of the outdoor work was done under a pavilion which further minimized any weather effects.

.	PSG-1	0.30g	.		M-4	0.25g	.
.	0'	20'	50'	70'	0'	15'	30'
Calculated	516.3	439.2	345.1	293.3	290.6	251.5	217.4
Observed	516.3	440.0	342.9	294.6	290.6	254.1	215.1
Std. Dev	-	4.7	4.1	5.2	-	1.7	2.8

Notice that the calculated data is well within two standard deviations of the measured results. In most cases, the test data was within one standard deviation of the calculations. The test data deviated by more than one standard deviation for only two out of more than thirty datasets.

Keep in mind that these measurements were taken both in and outdoors. While a crosswind will affect the BB's trajectory laterally, it will not alter the forward velocity of a BB in flight. Even so, testing was only done on completely calm days.

One thing that has been asked before is whether misalignment of the chronograph or rifle could lead to errors, as well as lighting conditions. When using the chronograph, I was always in the shade and used the lighting attachments for consistency. The same lighting attachments were used indoors. Each time the chronographs were set up, they were also tested to make sure that they were registering readings that were consistent with one another (clocked using an AEP that consistently fired shots within 1% of average).

As for alignment, one of the guys I was working with asked if we needed to slant the chronographs to take into account the fact that the BB was moving both forward and downward when it passed through the down range gates. Actually, it is better to keep the chronograph level as we were only concerned with measuring velocity along the horizontal axis -- the vertical velocity would be completely dependent upon gravity and is easily calculated.

Additionally, the gun could be aimed off-angle to the chronographs. While it is easy to notice a misalignment that is five or more degrees, it's harder to see a misalignment of only one or two degrees. Fortunately, this type of misalignment does not contribute to great inaccuracies. Chrony chronographs are used by the Sheriff's Department here, and we were discussing with them bullets that were fired at around 2800 fps. A five degree misalignment would only introduce a 10 fps error, meaning that we would measure 2790 fps instead of 2800. If a rifle that normally fires 0.20g BB's at 325 fps were misaligned by 5 degrees, the erroneous reading would be only 324 fps -- considering that most rifles shoot over a range at least as wide as +/- 3 fps, a 1 fps difference is acceptable. Even a misalignment of 10 degrees, which is ridiculously noticeable, would show the 325 fps rifle as shooting at 320 fps -- quite an error but, again, if you aren't able to visually notice a 10 degree misalignment, you should probably put down the rifle and visit an ophthalmologist.

(Just as a side note, I do want to stress that at the latter distances, it was very difficult to take measurements. I found that at extreme distances -- greater than 50' or in the case of the PSG-1, greater than 80' -- I would only get a reading on the chronograph about once every 1-2 shots. If anyone makes similar measurements, please let me know. If you do collect some data, please also let me know what type of chronograph you were using, what altitude you were at, and the ambient air temperature.)

All text, images, and linked webpages are the property of cybersloth.org and may not be reproduced without permission. For more information, contact airsoft@cybersloth.org.

The Airsoft Trajectory Project