The Importance of Evaluating and Testing 3D Filament
We believe filament should not be a limiting factor when trying to print something, whether because of cost or from lack of quality, which is why we developed an evaluation process to determine the best manufacturer for our filament and make sure they remain consistent going forward. Below is a summary of the evaluation and research process we used in our quest to find the highest quality 3D filament for the most reasonable price.
Getting Started
We began by evaluating the filament provided by already established filament sellers. We saw a lot of good feedback about Diamond Age and Faberdashery on forums like RepRap, and based on our research, we agree that they produce some of the best quality filament but we knew there had to be a better solution that gave similarly great quality without costing as much. In other words, a filament that gave you the best overall value without sacrificing quality.
| Diamond Age | PrintBL (US Distributor for Diamond Age) |
Faberdashery | |
|---|---|---|---|
| Price Per Roll – 1.75mm PLA | |||
| 300m ~0.9kg, Spool | Not Offered | 300m ~0.9kg, Spool | |
| 1 Roll | $106.33 | N/A | $128.73 |
| 2 Rolls | $103.09N/A$116.80 | ||
| 3 Rolls | $96.26 | N/A | $111.76 |
| Price Per Roll – 3.0mm PLA | |||
| ~300m 1.0kg, Spool | ~300m 1.0kg, Spool | ~300m 1.0kg, Spool* | |
| 1 Roll | $79.25 | $59.35 | $83.27 |
| 2 Rolls | $65.24 | $55.73 | $69.71 |
| 3 Rolls | $60.54 | $55.49 | $63.98 |
| *We had to adjust the price to reflect 1kg since Faberdashery sells in 100m coils | |||
Notes:
- 1 USD to NZD on 5-16-14: $0.8638
- 1 USD to GBP on 5-16-14: $1.6806
- Spool: Wrapped around a core
- Roll/Coil: Not wrapped around anything
- Faberdashery: Sells in max 100m coil lots and currently has no U.S. distributor so it had to be ordered directly from the UK and the price was converted.
- Shipping destination ZIP code is 49002 (Portage, MI)
- Used the lowest cost shipping method
- Prices are based on 1kg (~300m) of Black 1.75mm PLA Filament
As you can see, both of these manufacturers are expensive, even the Diamond Age U.S. distributor still costs a lot even when purchasing 3 full rolls.
So we set out to find a better alternative. We knew we had to analyze each filament using a consistent standard that enabled us to rely on the data collected and evaluate each new shipment of filament once we had selected our manufacturer. We also wanted to approach each manufacturer with an open mind, to have no biases towards any one company, including the 2 mentioned above. After all, we were looking for filament for ourselves too!
What We Tested
We wanted to make sure that our methodology was unbiased, disciplined, and consistent, so we looked at 3 key points to develop our evaluation standards around:
- The consistency of roundness (ovality) and diameter
- The consistency and quality of the prints
- The consistency of color
Looking at Ovality & Diameter Consistency
We began by analyzing the filament with a laser micrometer. We had it set up as a 2 person team: the first pulls the filament through the jig so that it would pass through the laser, and the second records all of the data into a computer database. The filament is measured every 1 to 3 inches, and we recorded the measurement to a 0.001 mm tolerance with 100 data points to help minimize any statistical skewing of the data. This same process was repeated for all of the filaments we measured.
We then analyzed all 100 points of data, looking at several statistical factors. The two we felt were most important were consistency of roundness and diameter.
To measure roundness, also called ovality, we took the absolute difference between the X-axis and the Y-axis to generate a positive number for all outputs and then we compared the mean and median, the maximum and minimum numbers, and the standard deviation.
To measure diameter we took the average between the X-axis and Y-axis then ran the same analysis
Printing with the Filaments
As the saying goes “The proof of the pudding is in the eating”. The next step in our analysis is to see how the filament printed, in order to test the chemical makeup of the filament and see if it can do what it was designed for. If the filament didn’t have a good design on a chemical level it wouldn’t matter how consistent the diameter appeared.
We wanted to print an object that was easy to print but with enough complexity that we could see how each filament performed. We chose a gyroscope of a carbon atom, with 2 levels and a round core.
After each object was printed, we examined how much stringing was left, how easily the gyroscope would spin after removing from the build plate, the accuracy of the edges of the print, etc. We also held the gyroscope up to a light and looked through it to measure how transparent the filament was, especially for colors other than black.
The Consistency of Color
Finally, we measured the consistency of the filament color using Pantone’s Color Matching System as our standard. Just like paint or dye, we wanted the colors to be dependable with every new roll of filament.
We compared the transparency levels of each printed object to see how much dye was mixed into the filament. More transparency meant less dye used. Unless transparency was specifically requested, we knew a solid, opaque filament was preferable.
Our Testing Tools
First Steps
We first started with a digital micrometer by Mitutoyo (model: MDC-1” PJ) which are one of the best in the industry for measurement tools. The tool worked great but unfortunately it was too slow to measure the filament and we felt it created too much human error when measuring.
The key point of error from our measuring efforts was measuring the filament on 2 axes points (the X and Y axis) and still be at the exact same spot. We just couldn’t do it with what we felt was reliable precision. This led us to choose a new micrometer.
A Better Option
Next we tried a dual-head laser micrometer from Beta LaserMike. We had to purchase two parts: the scanner and a processor for $2,000. We sent both to Beta LaserMike to be inspected, to have a new chip set installed, and finally calibrated for another $620.00. We also purchased a calibration tool to be sure it remains accurate, which cost us another $650. This cost us around $3,400 for the entire setup to accurately measure filament.
LaserMike Specs:
- Measurement Range: 0.127 mm to 25.40 mm
- Resolution: 0.0002 mm
- Repeatability: +/- 0.0008 mm
- Linearity (@100 scans per second): +/- .0013 mm
This was a lot of money for us but we felt it would give us a solid footing to measure the filament, thanks to this micrometer’s highly accurate resolution, repeatability, and linearity. We felt it was more than enough to handle our requirements.
Once we had the laser micrometer set up and ready, we had to find a way to hold the filament in place so that the measurements were as accurate as possible.
We Used 3D Printing to Help Solve the Problem
We designed a 3D image of a jig that could hold the filament in a fixed location under the laser, and then printed it on our 3D Printer. It was a fun thought that we were printing an object that would then be used to measure itself. The jig is in 2 parts and fits together around the opening of the laser, with a hole just large enough for the filament to pass through without altering it or getting stuck.
The jig kept the filament in place within the laser beam, and kept it perfectly perpendicular to the laser to create the most accurate measurement of the diameter of the filament on two axes (X and Y).
Our 3D printer is a Type A Series 1 (Serial # 234) and we are using the standard 1.75mm head along with the standard base plate that comes with the unit. We used this printer to create the jig that measured the filament in the laser micrometer, as well as to test each filament after it had been measured to print a specific 3D object.
Pantone Color Matching System (PMS)
We knew reliable colors would be just as important to our customers as it is to us. After selecting our choice of manufacturer, we used Pantone as our industry standard for matching color batches. Each shipment of filament must match the Pantone colors we selected, so that our customers can expect to receive the same red, blue, green, etc, every time they order new rolls.
Sources of Error
Before we get into the results, we want to point out some possible sources of error that could be at play in this data. We feel we have done a sufficient job in minimizing these errors and don’t significantly affect the results. None-the-less, we have to be aware of them.
Linearity and Repeatability
Linearity is the consistency of accuracy (bias) over the range of measurement. As we understand, this is how well the measuring device stays true to the number it is claiming to measure, in other words the device itself is off not just its ability to measure something.
We helped reduce this source of error by 1) having the device professionally inspected and calibrated, 2) setting the linearity accuracy to 0.0013 and only taking measurements to 0.001, and 3) since we are using the data to create a comparative analysis between the filaments and not to the true value, if one filament had the error all of them would have the same error. We did make the presumption that the error in linearity was a constant.
Repeatability
Repeatability is the consistency of a single appraiser to measure the same part multiple times with the same measurement system; it is related to the standard deviation of the measured values. The laser micrometer offers repeatability down to +/- 0.0008 mm and since we are taking data to 0.001 mm we felt that, in the worst case scenario, the number would be off by 0.001 mm. We felt comfortable with that possibility, but it is a potential error we want you to be aware of.
Human Error
The last source of error is us, humans, and our handling the filament. It is quite possible we could have input a data point wrong, altered the filament on a miniscule level with our fingers, or the jig itself could have altered the filament as it was pulled through the hole. We are making the presumption that these sources of error and negligible enough to not affect the final data and any error that does happen is consistent with all the filaments.
We set out to compare filaments with each other, and not to a true value, so if there were some minor errors in the system it would reflect in all of the filaments and therefore not favor one fitlament over others..
Our Testing Results
Below are the test results we were able to obtain from our analysis of 15 different filament companies. We compared roundness (i.e. ovality) and diameter consistency. We then took the place score for each filament and added them together to get a final result.
| Standard Deviation of Diameter Variance (i.e. Roundness or Ovality) |
|||
|---|---|---|---|
| Ranking | Manufacturer | Variation | % More |
| 1 | Diamond Age | 0.001670 | |
| 2 | Quantum 3D | 0.004006 | 139.89% |
| 3 | ESUN | 0.004174 | 149.92% |
| 4 | Filaco | 0.004737 | 183.67% |
| 5 | Coex | 0.005649 | 238.27% |
| 6 | Faberdashery | 0.006322 | 278.57% |
| 7 | MakerBot | 0.006614 | 296.04% |
| 8 | Singstek | 0.006732 | 303.10% |
| 9 | Type A | 0.007045 | 321.87% |
| 10 | Protoparadigm | 0.008542 | 411.51% |
| 11 | Jet | 0.008943 | 435.50% |
| 12 | Winbo | 0.009544 | 471.49% |
| 13 | JustPLA | 0.013785 | 725.44% |
| 14 | Delta Maker | 0.015092 | 803.69% |
| 15 | Zen Tool Works | 0.019963 | 1095.38% |
| Standard Deviation of Average Diameter (i.e. Consistency of size) |
|||
|---|---|---|---|
| Ranking | Manufacturer | Variation | % More |
| 1 | Faberdashery | 0.005202 | |
| 2 | Quantum 3D | 0.005705 | 9.68% |
| 3 | Zen Tool Works | 0.009385 | 80.41% |
| 4 | Diamond Age | 0.009452 | 81.72% |
| 5 | Jet | 0.010525 | 102.34% |
| 6 | Protoparadigm | 0.010595 | 103.69% |
| 7 | Coex | 0.010784 | 107.31% |
| 8 | MakerBot | 0.011775 | 126.38% |
| 9 | Delta Maker | 0.012734 | 144.81% |
| 10 | ESUN | 0.013175 | 153.29% |
| 11 | Singstek | 0.013685 | 163.08% |
| 12 | Type A | 0.014454 | 177.87% |
| 13 | Winbo | 0.017434 | 235.16% |
| 14 | Filaco | 0.023999 | 361.38% |
| 15 | JustPLA | 0.026105 | 401.85% |
Combined Rankings from Both Measurements Above:
Below are the final scores for each filament by combining both scores above. For example, if filament A was in 4th place for roundness and 5th place for diameter consistency it would have a final score of 9. We then sorted all filament brands by this final score.
| Combined Score | Manufacturer |
|---|---|
| 4 | Quantum 3D |
| 5 | Diamond Age |
| 7 | Faberdashery |
| 12 | Coex |
| 13 | ESUN |
| 15 | Makerbot |
| 16 | Jet |
| 16 | Protoparadigm |
| 18 | Filaco |
| 18 | Zen Tool Works |
| 19 | Singstek |
| 21 | Type A |
| 23 | Delta Maker |
| 25 | Winbo |
| 28 | JustPLA |
Conclusion
Based upon this data we were able to select a manufacturer that we could partner with who demonstrated consistency with the roundness, diameter, and color of the filament. We then started working with them to help improve the system even more and product colors that we felt would be most attractive for 3D printing.
We stand behind our process and the filament it produces, and it is our sincere hope that you feel the same way. We understand this industry is new, especially the personal 3D printing market, and there will be many new developments that we can all learn from to make an even better filament. We welcome your feedback and input on how we might make a better 3D filament.
Here’s What You Need To Do Next:
Try a test sample of our 3D filament. Choose one of out 10 different colors and decide which is your favorite for buying a whole roll.
Or if you are ready to buy, you can begin shopping and experience our 3D filament for yourself.
