join

Results You Can Trust – The Best Way to Test Golf Clubs

Post image for Results You Can Trust – The Best Way to Test Golf Clubs

Written By: Tony Covey

Can You Really Have the Best Golf Club Testing System Without Using Robots?

It has been our goal from day 1 at MyGolfSpy to create the best golf club review process on the planet. We thought if we use a range of handicaps, swing speeds, and swing types, provide more data, more detail, and be more analytic than anyone else, we could do just that.

I believe we have.

Using humans, imperfect as most of you are, to test golf clubs was a no-brainer for us. We've put hundreds of hours into developing, tweaking, and refining our review process. We spent hours on the phone with designers, engineers, performance specialists and club fitters trying to make our process even better. At every step of the way we were certain that, despite a total lack of consistency (even the best players in the world can’t touch a robot for repeatable precision), golf clubs that are played by humans need to be tested by humans.

Duh.

The reality is humans, play golf, not robots. Until we reach a point where golf is a game played by machines while their human owners drink at the bar, we’ll keep relying on imperfect humans to provide the data for our club reviews.

Golf is not Real Steel.

It may sound arbitrary...even wrong, but I assure you, the decision to rely exclusively on humans was not made without doing plenty of homework.

The Voices of Dissent

Not everyone agrees. The one perpetual sticking point with our process...the one issue that gets raised more than any other is our reliance on humans to provide the data for our reviews.

I can’t count the number of times a reader has told us that without robots our review process can’t be trusted. Despite our rigorous and consistent testing protocols, and our performance formulas which take into account things the average reader might never consider, the argument persists.

We've heard it time and time again.

  • Humans are too inconsistent.
  • Humans are unreliable.
  • You should be using robots.
  • Your results are invalid.

Only the consistency of a robot can tell you how well a golf club really performs.

Human results, they say, are almost meaningless.

The truth of the matter is that almost from the very first moment the "robot issue" was raised we starting looking into it. If it truly made sense to use robots for our testing, we'd find a way to make it happen. We did our research, we talked to experts from across the industry, and at every step of the way there was complete agreement.

When it comes to evaluating how a golf club will perform on the golf course, robot results, we were told time and time again, are almost meaningless.

We've shared bits and pieces of those findings in our reviews, but we've deiced that in the interest of putting the robot issue to bed and once and for all we'd lay it all out for you right here.

When the goal is to determine how a golf club will perform on the golf course, data collected from human testers without question, provides the most reliable insight into actual real-world performance.

The Problem With Humans

There are countless reasons why golf companies use robots to test the performance of their golf clubs and golf balls. Humans are inconsistent. We get tired. We have bad days where we can’t hit the ball to save our lives. Sometimes we hit it on the toe. Sometimes we hit it on the heel. Occasionally we even find the sweet spot.

We are…human.

A robot can hit balls a day long without a single bad swing. Robots don’t sweat. They don’t get tired. A robot hits the ball precisely where it’s told.

There isn’t a single golf robot on the planet that’s got a bug up its metal ass about TaylorMade, or Callaway, or anybody else that some real humans really love to hate.

The robot is always as objective. That is to say the robot is always as objective as the guys programming it and interpreting the data.

Robots can do things humans can’t.

“We can hit 3/4” toe shots over and over again and see how both the ball and the club reacts…Robots allow us to test the same club with the same swing, but at different swing speeds.  We can also just vary launch angle, as an example, with robots to see where a particular club may excel or not as well”. – Mike Vrska, Global Director of Golf R&D, Wilson Golf

Try doing any of that with a pathetic human.

As design tools robots are indispensable. They are essential.

If you need to test a specific impact location, or find out exactly how a change in CG placement from one location to another changes launch parameters, you’re going to need a robot. We’re not disputing that.

“We look at launch conditions up and down and all across the face.  This data is used to tune bulge and roll radii, face thicknesses, structural and mass properties”  - Nate Radcliffe, Director of Engineering for Nike Golf

The thing is, at MyGolfSpy we’re not in the club design business.

The Problem With Robots

There are some things – some more obvious than others – that you simply can’t get from a robot. For any aspect of design and performance that is not absolutely quantifiable, robots are absolutely useless.

“Robots are not good for heads when it comes to sound/feel, adjustability, workability, left/right directional tendencies, shapes, and visual aspects of setup like face angle, crown decals etc.”. –Tom Olsavsky, Senior Director of Product Creation for Metalwoods, TaylorMade-adidas Golf

Olsavsky certainly isn’t alone in his assertions that the human element is indispensable ingredient in the club design process.

“Only players can tell us if a new S-Flex shaft is too whippy or feels like a board.  Only players can tell us that a Fairway Wood feels great, except on heel hits when there is a weird sound or vibration on that impact.  Only players can tell us if the topline looks too thick or too thin. All of the ball speed, launch angle and spin rate data from players is very, very helpful from a design and development perspective, but talking with them during and after the testing can be just as valuable.” –Mike Vrska, Global Director of Golf R&D, Wilson Golf

The fact of the matter is that we don’t concern ourselves too much with that sort of subjective stuff either. Sound and feel issues should be worked out before the club hits retail (you know…during the design phase). Other stuff, shape, and color, and decals; there are no absolutes. As a golf company you design for your audience and hope for the best.

What matters most  – the reason why almost everyone relies so much on human testing (despite the fallibility of humans) – is that even the most advanced incarnation of Iron Byron doesn’t swing like a human, and as a result, doesn’t produce the same results that humans do.

Humans vs. Robots

Guys...especially the YOU NEED A ROBOT crowd let's take a brief timeout so you can prepare yourself for what's going to happen next.

Lots of really smart guys, guys who know more about testing golf clubs than probably any other group of guys on the planet - guys (including the guy who basically invented the golf robot) are about to tell you, without reservation or hesitation, that when it comes to determining how a golf club will actually perform on a golf course - robots ain't got nothing on humans.

One of the first people we spoke with about robot testing was Dick De La Cruz. For those who don’t know De La Cruz, he’s the innovative mastermind behind some of the most famous club designs in history. He’s one of the founders of Hickory Stick Golf (which ultimately became Callaway Golf). He developed many of the tools and gauges golf companies use today, and as it happens, he helped design and bring to life the modern swing robot.

De La Cruz understands that because of their precision, many golfers consider robots “the ultimate answer” when it comes to club testing, but he also tells us that in his experience, there aren't many people who actually know much about how robots work.

The problem, as De La Cruz explains, is that robot and human swings are fundamentally very different.

“To begin with a human uses two arms 2 hands and two wrists when swinging a golf club while standing on two legs. A robot has 1 arm, 1 wrist, 1 hand and stands on four legs. The robot is fabricated from metal and is very rigid as a human is very supple and flexible in the golf swing”. - Dick De La Cruz

I know…it’s all very obvious, and your first instinct might be to say “so what, robots are still more awesomer, and your data is still invalid”, but as it happens, those basic differences that Dick De La Cruz just spelled out for you...they make all the difference in the world.

Simply put, it’s not a question of consistency and precision, and knowing what happens when you hit the ball .73mm from the sweet spot; even when all other variables are equal, humans and robots produce different results.

The biggest shortcoming [of robot testing] is it is possible to get great results on the robot and then a club doesn’t perform as well with players. -Mike Vrska, Global Director of Golf R&D, Wilson Golf

We asked club designer and custom fitting expert Tom Wishon to explain why it is that a machine engineered to mimic the human golf swing produces results so inconsistent with those produced by actual humans. This is what he told us:

“Robots do not swing like humans in one VERY key aspect of the golf swing.   With humans, once they unhinge the wrist cock release, the arms begin to slow down as the clubhead accelerates to its max speed.  This happens because when the wrist cock is unhinged, that action sends the energy of the arms to the club.  With the energy of the arms sent to the club, the arms then have no other recourse but to slow down, while the clubhead after receiving the energy of the arms begins to accelerate to its max speed.

This action of the arms slowing down while the hands are still firmly holding the grip causes the mass of the clubhead to push the shaft into a forward bend position.  This forward flexing of the shaft causes the dynamic loft (the actual loft of the head at impact) of the head to increase and thus bring about the shaft's contribution to the launch angle, spin and trajectory of the shot.

The typical swing robot's "arm" is mechanically driven all the way through the swing.  Therefore when the robot unhinges its "wrist cock angle" the arms do NOT slow down as they do in a human.  This means that the shaft cannot act the same manner of forward bending as it will with a human.  And from this, the dynamic loft will not be the same for a robot swing as it will be for a human swing.  That in turn means the launch angle, spin and trajectory won't be the same for a robot hit shot as with a human hit shot.

Added to this is the fact that the robot's "hands" are completely rigid because the "hands" are made from metal and not flesh, tissue and muscle.  In humans the forward bending action of the shaft is dampened by the fleshy and supple makeup of the hands.  Not so in robots, so this too affects how the clubhead is delivered to the ball and changes the shot result of a robot vs a human. ” – Tom Wishon, Owner Tom Wishon Golf Technology

Say that again Tom...

"That in turn means the launch angle, spin and trajectory won't be the same for a robot hit shot as with a human hit shot."

And that brings me to one of the most fundamental issues with robot testing:

Robots Can't Test Shafts

This isn’t my opinion; it’s the well-educated opinion of TaylorMade’s Tom Olsavsky, Callaway’s Luke Williams, Tom Wishon, and basically anyone who actually knows anything about testing golf equipment with robots.

I think we'd all agree that the shaft is a pretty big and important piece of the golf club, and as it turns out, robots are all but useless when it comes to shaft testing. Robots show almost zero distinction between shafts of different flex, weight, torque, or any of the other engineering details that go into shaft design and ultimately play a role in how they perform in human hands.

It begs the question; if you can’t test the shaft, how do you test the finished club?

You might as well just stick a rigid steel rod in everything you test...which is exactly what Tom Wishon does when he tests his heads.

The completely rigid rod prevents the robot’s arm action from creating any atypical results. It allows Wishon to collect valuable information about the performance characteristics of his designs. What it doesn't tell him is how the club will perform for actual humans.

For Wishon's work as a designer, that's fine...probably perfect even. Robots are invaluable tools in the design process, but for the average guy who wants an idea of how a fully assembled, off-the-rack driver (yes...the reality is the average guy still buys off the rack) will perform for him, the robot data, it turns out, isn't particularly helpful.

We Don’t Buy Heads, We Buy Clubs

The average golfer also probably isn’t interested in having his next driver outfitted with a rigid steel shaft. The average golfer isn’t buying a head; he’s buying a complete club.

While a robot can tell you how a given head will perform at a certain speed, to a man, the experts we spoke with agree that it can’t tell you exactly how the assembled club will perform in human hands.

For that you need humans – living, breathing, wholly imperfect humans.

Robots Don’t Buy Clubs, and They Don’t Play Golf

In the last several years I've had countless casual conversations with designers, engineers, and other performance and fitting specialists. I've asked nearly every last one of them which - human or robot - yields more valuable information about the actual performance of their golf clubs.

To date, not a single person has answered "robot". Not one.

It's Actually Very Simple

Finally, we asked the experts a very simple question:

Which method of testing provides the best indicator of how a club will perform on the golf course?

Here’s what they told us:

Don't take our word for it...take it from the experts...all of them. When it comes to determining how a golf club will perform for an actual human, robots are a poor substitute for actual humans, which is exactly why MyGolfSpy has and will continue to test golf clubs using real live human testers.

 

{ 44 comments… read them below or add one }

Chris August 15, 2013 at 1:08 pm

Very good information but I always thought it was common sense that robots serve a purpose and in the end it is up to the individual to decide what works best for them. Similar to testing a club on a simulator or outside. You can find the club that feels best on a simulator but until you get it outside who knows what that ball is actually going to do.
I think the best reason to use a robot is to test durability. That robot can swing fast and hard all day without needing a break. You could let it swing until the club broke. That would give you a great idea on life expectancy in normal conditions. Trying that with a human might not be as efficient.
Maybe they can make a portable robot that I can push around the course and have him hit my shots? Just dial in distance, set it up on a line and let them swing away. I’m in for that!

Reply

Tom Wishon August 15, 2013 at 1:36 pm

CHRIS
A robot can be used for durability testing for sure, but it is far faster and as reliable to do your durability testing with an air cannon repeatedly firing balls at the clubhead. Typical way this is done is to hold the club by its grip with the head suspended in space. A laser on the air cannon quickly pinpoints the place on the face you wish to shoot the balls to contact. The air cannon can be dialed up to any ball speed you wish.

Typical initial basic durability testing consists of firing 5,000 shots at the center of the face at 125mph and checking the head after each 500 shots to note any changes in bulge, roll or weld lines. Other durability tests shooting the ball off center or on edges of the face can also be done, but the primary one is center face for 5,000 hits. Trying to use a robot to hit 5,000 shots would take a very long time compared to the bang, bang, bang speed of the air cannon test. And since the head is suspended freely in space for the test and not held rigid, the test of firing the ball into the head is going to tell you the same thing as swinging the club to hit a stationary ball.

Reply

Chris August 15, 2013 at 8:57 pm

Thanks for the info. I never thought about firing into a club face. I kind of thought the robot could go bang, bang, bang, quickly as well. Thanks

Reply

Mr_Theoo August 15, 2013 at 1:16 pm

YES! Finally proof that humans are better at testing than robots!

Reply

Slim August 15, 2013 at 2:21 pm

Any time Tom Wishon speaks, we should all listen…and then watch our collective heads explode with the knowledge he blesses us with. His discussions on shafts alone is better than any robot test could provide.

I commend MyGolfSpy for their detailed and informative club test. The data MGS provided on the 2013 driver test was amazing…and a lot better than some sort of gold, silver medal type rating. Crazy, real golfers with real golfer swing issues, not ‘in bed’ with large golf companies, with actual data to prove a point. I wish they had the man-power and resources to do those tests every year with drivers, irons and wedges. Speaking of which, need any volunteers to assist in the testing?!

Reply

TheHacker August 17, 2013 at 1:00 am

Every time experts like Tom weighs in on an issue, we should listen.

That said experts have been wrong before, at least when it comes to what the consumers really want.

Reply

MikeB August 15, 2013 at 4:41 pm

I’ve always thought that human testing was the preferred way of evaluating golf clubs. My guess is we won’t hear much about the robot testing group as they probably won’t even read this article. The closed minded people don’t want any information that is contrary to their beliefs.

Reply

flaglfr August 15, 2013 at 4:42 pm

It is nice to see that human testing is coming back into the equation. As has been elegantly said by several above., Robots do not play golf on a course. Unless of course the Governator takes to the game. Sorry…. Couldn’t resist that one.

It is VERY nice to see testing that relies on our frailties. Yes, engineering comes into play from a design standpoint, but in the end it is we living, breathing humans that whack the ball around the course. I believe it is of greater value to have testing that relies on we who do not hit it every day in a spot on the club smaller than a dime.

And if I see robots taking up the game, I am moving to hockey.

Reply

Barbajo August 15, 2013 at 6:22 pm

Spot on and well researched! To me it’s just common sense to have real live humans do the testing, but the Wishon and others bring serious credibility to the discussion. Thanks!

PS: My robot is going to be pissed, though. He was waiting for the next club test contest…

Reply

Charlie August 15, 2013 at 6:36 pm

Is any testing done with off the shelf clubs purchased from various retail outlets like Dicks Sporting Goods? That is what the AVERAGE guy is buying. What is the loft, lie, shaft quality, head quality and on and on? Give me the results based on testing those clubs.

Quality testing in a lab is one thing but when a product hits the mass production line it takes on a whole new meaning.

So, at best I go for human testing because I am not a robot.

Reply

John Vu August 15, 2013 at 7:21 pm

Before reading this article I thought this was going to be a guide on testing clubs. i.e. When testing drivers against one another do the following:
- 20 swings per driver
- using a launch monitor, aim for optimal ball launch/ball speeds of….
- etc etc

Reply

jp August 15, 2013 at 8:44 pm

Results you can trust? I struggle trusting results showing such precision (2 decimal places) when there is so much statistical variation in the data. Is that level of precision valid?

I see this as a debate between real world and lab results. BOTH are informative.

Reply

tiger168 August 15, 2013 at 9:47 pm

This is a great article.

I would like to see the robot use to “compare” clubs, not, so much for designing. For u consumers to make a decision to “purchase” a club with $400+ investment. 10 “GOLD” drivers doesn’t tell me which one matches my swing profile the best. And why bother with “SILVER” rate clubs. But, these silver rated clubs by human, would be interesting to put on the robot to see their objective number results? As many has surprised from last season of late that the Wilson D100 actually out perform many “GOLD” rated drivers. As later, the Wilson D100 mysteriously became a “GOLD” club.

The article would have been more helpful to the consumers for testing, is all I am saying. As one of the golf retailer is doing secretly. But, why secretly? I guess they don’t want to lose the chance of selling all across the board. And comprehensive testing of all available clubs are just economically prohibitive. Thus, advantage: golf manufactures. Whew!!! I can’t imagine if everyone rush out the door to purchase Japanese clubs only, since their manufacture specs are much tighter then the US club manufactures.

I had the luxury of watching a top pro picking his woods for the season. Boy, he literally went through more than a hundred clubs through weeks of trying to finalize on “10″ that he liked, and then, weeks later, narrowed down to “not liking any of them” and start the process again. It was an amazing experience. Oh, that include launch monitor, all the measurements of all the clubs he hit.

One of the toughest thing to pick the right club even after experts do the best fitting possible for the pro was that, the pro, as any human being, actually adapt to the club naturally, even the club spec is off. The pros now a days are so good, even if the club is off on spec, they can still make it work. It is a natural instinct of the pro, but, it sure adds another level of complexity to the fitting process.

Thus, robot is the more objective source for “compare” and “test”.

Thanks for a great article and provoking thoughts…

Reply

Harry August 15, 2013 at 11:11 pm

I just bought a new set of clubs and the fitter decided that Wishon clubs were the way to go for me despite the fact the fitter has his own range of components.
Having been told that was to be the clubs I was getting I looked into Tom Wishon and his thoughts on golf equipment and he makes so much sense.
The clubs by the way are fantastic and if ever I get my swing consistent then watch out golfers

Reply

Joe Golfer August 15, 2013 at 11:39 pm

Great article. I had never considered some of the reasons listed as to why robot testing doesn’t work, such as shaft testing, among other things…

Reply

TheHacker August 16, 2013 at 9:25 am

I’m going out on a limp to disagree with the experts. While it’s reasonable to say that robots don’t buy clubs, and they don’t play golf either. But likewise, those lucky few MGS human testers don’t swing the clubs for me, nor play golf on my behalf.

At the end of the day, I have to rely on testers who don’t swing like me, play like me, or have the same imperfect swing like me to test the clubs which gives varying data that suggest whether the club might be the best for me. Its not that much better than asking some bloke in the range who happens to use a new driver how it faired for him. When it comes to opinion such as this, anybody’s is as good as anybody, and nobody.

I think part of good consumer information is to take away as many variables as possible, at least provide a baseline across the board. Say with a typical 18 hcp swing of 95 mph (or a 105 mph of a mid hcp), a level angle of approach, perfect contact, how does club A compare to club B? This at least tells us uninformed consumers who may never have a chance to try out the club ourselves in the range then at the course, which club is the best bet.

After you have that baseline information, then have human testers to test it out with their swing. And publish all the information robot and human testing results. I still think it is as much a fallacy to say that human testing is better than robots, as it is the other way. If robots were useless, why does every major manufacturer use one? At the very lest, if it provides a consistent baseline, that alone is a very useful piece of information.

Reply

golfercraig August 16, 2013 at 10:03 am

So, I can’t trust the data from a robot, but i can totally trust it from humans who don’t swing like me? Who have a definition of “feel” totally different from me? Got it.

Test your own stuff. Anything else is a ploy for pageviews. Find a shop/pro that will let you hit, and then BUY YOUR STUFF THERE. Don’t use them as a fitting room at a department store while you run to eBay to get the product.

Reply

TheHacker August 17, 2013 at 1:14 am

Dear Golfercraig,

There’s no such thing as a review by anybody that’s a perfect substitute for testing stuff yourself. It’s never possible. However where I come from, we do not have very professional nor respectable club fitting professionals. Our retail stores only carry shaft flexes up to S-flex, and everything has to be shrink wrapped nicely because clubs are rather expensive items here.

At best there’s an indoor driving net for you to take a few swings before the sales clerk come over to ask you “So are you buying it or not?”

What consumers like me need, is as much inputs as possible, and this article rather conveniently quote a few experts to dispel the value of robotic testing. However I feel it has not address the elephant in the room issue – how do you scientifically compare the characteristics of one club to another when human testing is so inconsistent? At best, human testing might be one good source of input, but totally debunking the value of robotic testing is depriving sincere consumers like me from another very valuable source of input.

Which is why I come in so much to debate the issue, despite the overwhelming weigh of expert opinion against robotic testing. My point is, granted robotic testing does not fully replicate the dynamics of human swing, it still provide very valuable and objective data as to how one club performs with a certain set of parameters compared to another with the same parameters. That I strongly believe has to be of some value for consumers, just like MPG rating on cars, or electrical consumption indicators for air-cons and fridges. It is useful – even valuable consumer information.

Reply

Sam August 16, 2013 at 10:56 am

Callaway nails it here. Humans play golf. Humans should test products. Robots should be used during certain phases of development and to QC products, to see how a ball off the heel/toe/high/low/anywhere on the face is going to. But then you need the final product delivered to a human for testing, period.

Reply

Tom Wishon August 16, 2013 at 10:59 am

It is obvious from the various comments pro and con that what golfers want is some assurance that when they spend a significant amount of money for a new club or clubs, that the clubs are going to perform well for them so they get their money’s worth.

For many golfers who do not have experience in club design and club performance research, it is only obvious to think that a robot hit test will provide this with reliable accuracy. That’s logical to think that because robots swing the same way every time. Then you have other golfers who think that they can gain this assurance before buying from human testing, because after all, golfers are humans, not robots.

The real truth in this is that neither robot nor human group testing will ever be able to give any golfer the assurance that what they spend their money to buy is without question the best clubs for them. Robots don’t swing like humans and cannot make the shaft perform in the club as it does for humans. And human tests don’t do it either because you have no idea if your swing characteristics are the same as those of any of the humans doing the hit tests. Only if the companies offering the human test results would also include videos and swing measurements of the golfers in the test could you possibly come close to knowing if you and your swing are going to get the same or different results with the club(s) that the humans did.

That all boils down to this. If each golfer’s goal is to have the assurance that the money they spend will result in the very best performing golf clubs for them and their swing, there is one and only one realistic way to do that.

And it isn’t going to a store, hitting every single club in the store and then taking the ones that show promise for your swing out to the range and out to play a round for 2-3 sessions over the period of a week. That would work. But think about that – what retail store or pro shop is likely to let you do that. Maybe if you know the store manager or the pro very closely you could arrange that. But that would be rare.

So the only way you can come closest to knowing that what you are about to fork over your money to buy IS the best for you and your swing is to go through a serious fitting session with a GOOD, EXPERIENCED custom clubfitter.

Yah, yah, yah – I can already hear some of the comments a few might make to this statement about me shilling for custom fitting because I own a company that designs models for custom clubmakers. Fine for those who want to think that.

But use your brains – if you want assurance that the clubs you plan to buy are the best for YOU, you have to be able to find someone who can first figure out what you need to best match YOUR unique combination of size, strength, athletic ability and swing characteristics. Then you need to hit combinations of test clubs based on that analysis until the one(s) are found which you and the clubmaker determine are best for YOU. And then you need to be able to have the option of having the clubs tweaked to fine tune them after the fitting.

If you really think about it, that’s only going to happen with the more experienced custom clubfitters. Sure, not all who say they are a clubfitter can really do this with proficiency. But there’s a heckuva lot more in this country who can do that than there are sales people at retail stores and pro shops who can do this. You’ll have to dig and do research to find the better clubfitters out there and some of you won’t find such a person close to where you live. But they do exist and they are your best bet if your goal is to be more assured that what you buy is going to work the best for YOU.

Sorry if my comments offend anyone. But this is the absolute truth in this matter if again, your goal is to be assured that your money has the best chance of being well spent to get what you want.

TOM

Reply

Stu August 16, 2013 at 1:39 pm

Agreed 100%.

Reply

tiger168 August 16, 2013 at 2:16 pm

Hi Tom,

Thanks for sharing your thoughts.

I like to inject that concept of the robot testing of club (my assumptions/wishes, maybe naive, but, however, it is more practical), for off the shelf or fitted clubs, isn’t to duplicate the test to the human swing at all. I know human cannot duplicate robot and vice versa.

The concept will be let’s say, my driver swing speed is 110 mph, blah blah blah. The retailer will say, you need a STIFF, the fitter will do some more work and say, your maximum is 45.25 inch, 8.75 degree and a Bridgestone 330-RXB on a cold day. Let’s just use that.

What I like to see a robot test is to a “range” of results, base on the max profile and an average profile using the selected/fitted club. For example:

With a typical graph scenario, having x, y and z axis.

With the y-axis bearing a range of swing speed from 90-115 step of 5 mph, since I max out at 110 mph; x-axis as progressive launch angle from 10-16 step of 0.5 degree at a time (Since when you work the balls left or right, you want to see the close-open club face effect); then the z-axis, one I think being most important, is mapping the ball impact position on the actual club face (middle, 0.25-1.00 inch to the left and right; up on the face, middle, and 0.25-1.00 inch to the left and right on the top; lower on the face, middle and 0.25-1.00 inch to the left and right on the lower face). Then plot the dispersion and the distance separately and together, the result should be a “spherical area”, where I can measure my 70%-100% hit rate and average distance as result from a particular club, in this case the driver.

This method, as I have commented earlier, are probably economically prohibitive, but, it’s the true effective robot test I would love to see for the potential clubs that might be fitted for me. Off the shelf or custom.

I am trying using my own playing data to do this, as I play an average of 2 rounds of golf per week, and you only hit driver probably a good 10 holes, and even that, is only 20 data points per week in a variety of shot setup (left to right, right to left, high/low trajectory). That is why a robot test of this sort will make a lot of sense for me.

I will pay $150 for any new driver (because they all eventually come down to that number), but, I will pay $300 minimum for this type of test result, which will put me in the $450 range, and I know I will have made a best guesstimation by far.

Then I will know what driver to bring to the tournament, and which one to bring on the weekend for fun.

Thanks Tom, pls keep up the good work and really appreciate your article and additional comments.

Reply

Tom Wishon August 16, 2013 at 3:55 pm

Tiger168

I am sorry to say this, but no matter how specific, how detailed or how controlled one makes a robot hit test, there is just no possible way that data could be used to make concrete conclusions for how the club(s) will play in the hands of a human. It is utterly and completely impossible no matter how much you try to control all the variables to do this. All clubs have to have a shaft to be hit. No robot can make the shaft perform the same way it will for any human golf swing for the reasons stated in the article. Since the shaft has a real effect on launch angle, spin, trajectory there is just no way a robot can offer valid data that can be compared to what a human will achieve with the same club. I’m sorry but this is just the way it is because robots can’t hit a club the way humans do.

So the only way you’ll find that driver to bring to the tournament is if you get perfectly fit by a really good clubmaker who ALSO will let you hit test clubs over different playing and range sessions until you find that one that is best.

TOM

Reply

TheHacker August 16, 2013 at 8:27 pm

Dear Tom,

I might agree with you that robot testing provides limited utility for decision making, especially when it regards to deciding which is the best driver to to buy. You are the expert and who am I to argue that?

That said, it provides very meaningful comparison for us as consumers, when there is a constant baseline with which one can compare brand A with brand B, with exactly the same swing which one is the longest. You can test the difference for 95, 100, 105, 110, 115 and 120 mph swing, I am sure we’ll be able to uncover some interesting figures.

You see, manufacturers will NEVER release such figures, nor will they agree to a common baseline or test perimeters. But if we compare everyone of them, same swing, centered face contact, level angle of attack, square face, straight head-on contact, with this same perimeter across all the brands, how does club A stack up against club B, or other popular brands in the market?

I am sure most of us here are not so foolish as to base our buying decisions on just one factor – like robot testing. Its just good consumer information which is absent everywhere in the world. It is just like the fuel consumption information on cars, there is (some sort) of a baseline test to come up with an MPG rating for the car. Although different people have different driving habits, and you encounter different traffic conditions that affects fuel consumption of the car, it is still a very useful and important consumer information.

Take it as a baseline measurement, how does one club stack up against another? Nobody can give exactly the same swing when he switches from testing club A to club B. That is why at the very least, human testing is not that much better than robot testing. At best they are just as good as each other, but still in the end, we have neither the robots nor the MGS testers playing golf on our behalf. What fun would that be anyway?

What I am asking for is good baseline information, whereby you can compare club A to club B, with every other factor being held constant. Its not going to be the one deciding factor when people buy their clubs – give us some credit as we are not that dumb. Its just one useful input to consider before we put down good money for golf equipment.

Reply

tiger168 August 17, 2013 at 12:21 am

Hi Tom,

Thanks for taking the time to respond.

YOU MISSED MY POINT… I think…

You see, I am not really concern about manufacture design or robotic test during that process, if they use the robot in their process, GREAT! If they don’t, GREAT!! That was not my focus of my version of wishful robotic test.

I agree 100% human cannot repeat themselves in their swing, let alone duplicating that in robot. I also understand, manufacture cannot produce two graphite shaft with 100% identical characteristic/spec, so do manufacturing head. even if they use precision aeronautic fabrication facilities and equipment. No two heads, no two shafts can be produced “identical, not to mention how they are assembled.

My point is to use robotic test after the fact, which is that the clubs are actually sitting on the shelf, or assembled by the fitters. AFTER!!! Using the methods described in my earlier comment. And I have only put down 50% of the criteria in that response. But, even that, to proof my point, is probably economically prohibitive, but, I was willing to pay for the results, that was my point.

This method will find the one “closest” to my profile, (ok, I can/will select it myself) and also filter out the “defect” during the manufacture/assemble process, if there is any face/soldering/glue/tipping/weight discrepancy/defect/variation and plot the sweet spot on the face/shaft length, wouldn’t that be wonderful???

After all, isn’t SPINE/PURE/FREQUENCY MATCHING/HARRISON INSERT/TIPPING are trying to do the same thing, but, with their own specific testing equipment? But, I wouldn’t want to tinker the club without the data collected properly (hint, hint, robot).

Thanks again for the thought provoking discussion.

Again, not asking the robot to do what the human do; or vice versa…

Reply

SKB7840 August 18, 2013 at 8:05 am

Tom,

I agree with what you are saying. I have asked the people from this site about robot testing just to try to narrow the field down. I wanted to know if the new driver head really is better than the old one, all conditions being the same. Or if there was truly a difference in any of the clubs other than aesthetics or shafts. My guess is that robot or drop a ball into the face that the Rocket Balls drivers probably new to old performed about the same.

So if the only way to be assured is to find a good experienced fitter, what is the best resource to find that person? I went to the “best” fitter in my area and I am not convinced that they did a great job. Am I looking for the guy that has his own components? Or the guy that has the fitting carts from every major company there is? I have been fit a few times over the years for various clubs and I have found a major difference in the methods.

Thanks,

Reply

Charlie August 16, 2013 at 11:22 am

Thanks Tom for the candid comment.

For free I had the golf pro where I golf measure swing speed and lie angle and had an OEM set of clubs made to those specs. Improved my game tremendously because of the accuracy the clubs gave me.

So, even if you can’t get a complete club fitting as Tom suggests maybe you can get the help I did.

Reply

Thehacker August 16, 2013 at 6:36 pm

Buying golf clubs is a gamble. No matter what the experts say, I feel that vendor neutral, unbiased test results have its place in the markets, and that’s exactly what the market lacks.

I have seen many club fitters that are more keen to fit me to what they have in their inventory that I end up more confused after the session, and I totally hate club fitters who “doubles” as a swing coach. That is why I will not go one anytime soon.

I think the experts in this forum will just have to accept that a large number of audience in here are engineers / scientific type who are just skeptical of qualitative information provided by humans. That is exactly why people are drawn to this site at the first place, we are cynics and do not believe other golf sites as being independent.

It nights be I am the most incorrigibly cynical person in earth, who will nod my head and agree with the experts as they make their point; but in the end we still like to look at the quantitative data, at least see if it collaborates what human testing says.

I’m sure most of us here would be curious how all these clubs stacked up if measure with a totally consistent robot. Was TM really BS about their 17 more yards claim? Was C hyperbolizing too much about being the king of all distances? Equipment junkies like to know such things.

If nothing else, that’s the reason why many of us come to visit this site.

Reply

Barry Obamant August 16, 2013 at 7:30 pm

I both agree and disagree with certain aspects of human testing. While I do agree that it is necessary to use real players to gauge certain subjective characteristics (look, feel, sound, etc.), the fact of the matter is that if you have a bad day, your performance results are going to suffer accordingly. The reported ball flight, distance, workability, etc. will be skewed. When reviewing the numbers of a club, in my mind, it is necessary to garner accurate numbers. For example, while a golfer may have a swing speed of 85 mph, his contact will not be perfectly in the center swing after swing. However, a robot could simply be set to swing at 85 mph, and then instructed to make contact in various areas of the clubface to provide accurate, reliable results. And some may say that robots cannot test things such as workability, it is not much more reliable to use humans. Many players can work the ball from left to right (or vice-versa) with clubs they have been using for a few years. But if you were to hand them a brand-new iron and give them ten shots to warm up before testing, the same player may be unable to hit so much as a baby draw. While I applaud MGS for all of their dedication to the system, I simply believe that in order to claim a superior testing system, robots must be included. Each golfer is not the same, and it is impossible to test the complete range of swing speeds and contact points without using some form of automatic, repeatable process.

Reply

Barry Obamant August 16, 2013 at 7:35 pm

One more thing. While I do value Tom’s opinion, I do feel that it is closed-minded to completely ignore the values of robot testing. Although the shaft does play a major factor in various aspects of ball flight, it is not true to state that a robot test without a shaft is completely useless when compared to a full club in human hands. Yes, the shaft can affect the flight of a ball, it will not affect data involving clubhead speed and contact.

Reply

TheHacker August 17, 2013 at 10:43 pm

I totally agree with you. It is closed-minded to completely ignore the values of robot testing, the issue won’t ever go away, and this is one matter that no matter what experts may say, many consumers will still put some weight on the value of robot testing results.

Yes, the experts can chime in, and give us their few cents worth of caveats, but still it is not a compelling enough reason to not have robotic test ever, and providing the information to readers. I feel this is all in the interest of fair and objective consumer information, which up till now has never been made available.

Reply

Tom Wishon August 19, 2013 at 10:22 am

BARRY
In the article, I said that I use robot testing in the design of my clubhead models. But to eliminate the shaft from affecting the data, I use a super rigid vitrually non bending shaft in the heads. Robot testing is done to compare the HEAD design elements on their own, free of any influence from the shafts. So in my robot testing I will be looking at the CG effect, MOI, face design of a new model vs known results for these elements on previous designs. To be able to robotically compare one head to the next, you have to eliminate all other variables in the test, one of which is the shaft. Hence to get the best “apples to apples” comparisons to really know if one head’s CG, MOI, face, etc is different than another, we do this by eliminating the shaft’s possible variations in this by using as rigid of a “shaft” as we can.

That to me as a long time designer and researcher in this industry is the only value of a robot. And really, since this type of more controlled testing can provide results to verify computer FEA modeling, the day is coming if not here already when really good event simulation modeling will eliminate the need for a robot hit test completely.

TOM

Reply

flaglfr August 17, 2013 at 8:33 pm

All,
Let me interject something that really needs to be brought back into the conversation…. Feel. There is only one way to get this. Human testing. A robot doesn’t feel shaft flex, impact point or anything else. This is a critical piece of why we play what we play whether we want to admit it or not. A club has to feel good to us for us to pick it and use it.
If a club doesn’t feel right to you, I can almost guarantee it will not find its way into your bag for very long if at all. The only true way to take advantage of feel is tobe properly fitted and then make the call based on feel. Yes it is subjective. Yes it is an unquantifiable charachteristic. But nonetheless, I would argue that this is one of the most important things we look for whether we realize it or not.
The only way to get a feel for feel is to have humans (golfers and good fitters) in the equation and (as much as possible) manufacturers out of it. This is why the way testing done on this site (IMHO) is so valuable.

Reply

TheHacker August 17, 2013 at 10:32 pm

Dear Flaglfr,

I agree with you about robots not being able to feel the club and describe it for you. However, feel is a very subjective thing. Having several testers play the club and describe the “feel” for you – whatever that mean, is not much better either.

The problem is feel means very different thing to different people, and different players perceive feel differently. Just like some people find chilli pepper tolerable, while others break into sweat when they try it.

Back to the whole deal about club testing, I still feel (pun unintended) that robot testing should form a very important part about club performance testing.

Reply

Foobar August 17, 2013 at 10:30 pm

Humans are really good at judging looks, feel, or sound either. Read reviews. Nobody agrees. Honestly the stats are pretty useless. The sample size is too small and the variance between golfers day to day is too large. Some of the shaft comparisons are nice in the sense they suggest what you might expect by changing flex/shaft combos but to pick a shaft off them would be crazy.

Go through the forums and read the user reviews of drivers. For pretty much every driver (Covert, 913, stage2, xhot, G25, amp cell,…) you will find some guy saying it is a total bomber and some one else saying it sucks. For whatever reason it fit the first guys swing and not the other guys. Over time you might find that you map closer to certain people/shafts (i.e. you like high launch/low spin) and use them to narrow down you choices.

And no robots are not going to help anything. You need to get out and swing the clubs. A good club fitter will help you narrow it down from 10k+ combos of heads and shafts to a more manageable number but at the end of a day picking clubs is pretty personal.

Reply

Dave S August 18, 2013 at 11:04 pm

Why not include robot testing as part of the overall testing? Wouldn’t that help lend more legitimacy to the process in the minds of most consumers? I believe that humans are still prob the best way to test, but think robot results are important as well. If they weren’t, golf companies wouldn’t shell out a ton of $$ for these machines and the engineers who run them…

Reply

Tom Wishon August 19, 2013 at 10:28 am

DAVE

Speaking for my own head design work, and I bet which is the same for the other companies, the only value of robot testing is to compare one head design to a previous one(s) so we know if and how the different CG, MOI, face design, etc brings about any change(s) in launch parameters and flight chracteristics. But to do this properly so that the test results are valid for the head’s design changes on their own, you need to eliminate the possible variable to this of the shaft. Hence in my case, this is why I use a completely rigid shaft so that there is the least possible bending of the shaft coming into impact. That way I get closest to “apples to apples” comparisons of the head design differences.

If you let the shaft get involved in anyway, whether through humans of different swing types making the same shaft perform differently coming into impact, or with robots making the shaft not perform like a human, then you introduce variables that confuse the results so you cannot make valid conclusions about the effect of whatever head design change you made on the head.

TOM

Reply

TheHacker August 23, 2013 at 11:23 am

I am rather surprised at the seeming reticence of the experts who spoke out against the value of robotic testing of golf clubs, in the face of numerous requests by consumers who desires to have such information available.

To us as consumers we like to know, because we are paying good money for golf clubs and having every information possible to be made available, whether or not is it going to be helpful in the end, at least we considered all the possible inputs. Maybe it’s cheaper to have human testing, as there is no lack of volunteers, compared to the cost of getting an iron byron type robot. If that’s the case I would understand.

But if its because the powers in MGS decides that we could do without the data obtained through objective robotic testing, because in theirs and the experts opinion that such data is not useful, then I really beg to have this reconsidered. Like I mentioned before, it’s all in the interest of providing as much information as possible for the consumers, just like providing MPG stats to car buyers. It is still a very much valued input that works together with user reviews and finally personal test drives.

Reply

Tom Wishon August 23, 2013 at 12:01 pm

Despite the fact we have tried to tell golfers the technical reasons why robot comparison testing of clubs is not going to tell golfers what they want to know, some who are commenting are choosing to ignore what we with the experience to know are trying to tell you. It’s like some are saying, “don’t confuse me with the facts, because my mind is already made up that I want to see robot hit comparisons as my way to decide what to buy.” We are simply trying to use our experience to help golfers the best we can.

The facts are that we DO want golfers to make the best buying decisions. When golfers decide to spend their money, we DO want golfers to spend their money on the RIGHT club(s) for their swing and manner of play. Robot hit comparisons cannot tell you exactly what you want to know to be able to spend your money for the best club(s) for your swing and manner of play.

Let’s talk more performance specifics.

What you want to see from robot comparison are chiefly the differences in launch angle, spin, ball speed/distance, dispersion/accuracy so you can know if this or that club will allow you to hit the ball longer, straighter, or more consistently.

Because of the significant differences between how a robot and a human make the shaft perform, and because the bending of the shaft most definitely affects launch angle and spin, robot comparisons cannot tell you with any degree of certainty how YOU will hit this or that club with respect to launch angle and spin.

Because of the fact there is a +/- tolerance in EVERY company’s driver head production for the final COR of the face that typically ranges from a COR of 0.810 to 0.830, if a robot test says this club had a higher ball speed and from it, more distance than these other clubs, that is absolutely no guarantee that when you go to the golf store to buy that driver that it will hit the ball with a higher ball speed and distance. You might happen to buy a driver on the + side of the COR tolerance just as you can buy one at the minus side. Distance comparisons from robot testing are worthless for being able to unequivocably guide any golfer to the longest hitting club.

And since it most definitely is each golfer’s own swing path and face delivery on the downswing that dictates accuracy, if a robot hits this or that club straighter or not, in no way can that tell you how YOU or any other golfer with your specific path and face delivery from your swing will hit the ball with regard to accuracy.

In the end, it is as I said in a previous comment to this discussion. The ONLY, and I mean capital ONLY, way that each golfer can be sure of having the highest possible chance of knowing that the club(s) he will buy are the best for HIM/HER is to find the best, most experienced custom clubfitter you possibly can, and put your needs in his/her hands. Do that and you will get the very best performing club(s) for YOU and YOUR swing.

Reading and reacting to robot or even human hit testing results most definitely will not do that. And really, the reason I am so passionately trying to tell everyone this is because I do care personally that golfers find the very best club(s) for their performance and money.

TOM

Reply

flaglfr August 24, 2013 at 5:47 pm

Hack,
Your theory is sort of like saying that understanding the construction of the stealth bomber is going to give you a better ride on the next commercial flight you take. Yes specs are a good thing to know. but how many “average golfers” even know what COR is? Modern fitting machines tell you things like clubhead speed, ball speed, backspin, etc. This in combination with an experienced clubfitter will tell you much more than a robot test. Yes it is also great and (if possible) a test drive of the clubs you are contemplating AFTER the fitting will be of the most benefit to all. The club will not only function better for them, but they will feel better about a purchase based on how they play instead of how a machine hits it.

Reply

John Vu September 9, 2013 at 6:35 pm

Tom – I’m planning on going on a fitting this weekend. If I’m not striking the ball particularly well, would you recommend coming back on another day for another fitting, or could an experienced golf pro still put me in a set that would fit my game?

Reply

Tom Wishon September 10, 2013 at 11:50 am

JOHN

From having done many educational seminars for the PGA club pros over the years, it is sad to say that very few teaching pros know enough about fitting to truly help golfers get the very best fit that will allow the golfer to play to the best of their ability. The PGA’s curriculum for membership training does not include even close to the best information to teach the club pros how to fit golfers accurately, I am sorry to say. So trying to get a good fit from a teaching pro is less likely to happen.

When it comes to what you are talking about, do please understand that while we all go through periods during which we don’t hit the ball as well as we do on other days, our primary swing characteristics on which a good fitting analysis focuses do not change. What changes is the range of mistakes within our learned swing characteristics.

For example, a golfer who has an outside in path never all of a sudden changes to a square or inside out path when he is having a bad time with his swing. What changes can be that his range of HOW outside in he is from swing to swing changes. Same for all of the other swing characteristics. If we release the club too early, we never all of a sudden release the club late. And so on.

So the swing characteristics that a good clubfitter would observe to make fitting recommendations don’t change to the point that a good day or bad day would cause the fitter to make a completely different assessment for what fitting specs you need.

From our fitting research I can tell you that it is also very possible to see situations in which a golfer’s present clubs are so poorly fit to his size, strength, athletic ability and swing characteristics that those poorly fit clubs are one of the causes for him having as severe of a swing in how he hits the ball from day to day. Drivers that are too long, fwy woods too long, having the wrong total weight and swingweight, having the wrong face angle on the woods, even having the wrong grip size are all basic elements in clubs which can cause more of a range in poor swinging periods with golfers because these things can and do make it more difficult for golfers to be more consistent with their swing motions.

But at the end of the day, YOU have to make that decision because if you go into the fitting not feeling right about the process, that’s not a good situation either for you or for the fitter. So if it makes you feel more confident about the fitting to wait, then do that.

TOM

Reply

John Vu September 10, 2013 at 1:59 pm

Thanks for the reply

Reply

BillyBillyBilly April 3, 2014 at 3:59 pm

Anyone know who to contact at golf spy about getting your equipment tested against the other brands?

Reply

Leave a Comment