My personal double blind, randomized trial of freeform lenses

[ilanh]

To sort out the hyped up claims of the various progressives I decided to compare 6 freeform designs: Accolade Freedom, Reveal freeform, Shamir Autograph II, Ideal advanced, Definity and Physio 360. I realize that there are some semantic issues involved here regarding true freeforms etc, but these are the lenses that I had access to. The lenses were all fit into identical frames, using the same RX, PD, seg height in all cases. Polycarbonate with crizal alize was used in every case. Every frame was fit to my face in an identical manner (as confirmed by markings). Over a period of one month I used these frames interchangeably for reading, computer and driving. I kept notes on how well they did for each of these tasks. At no time did I know which lens design I was using since the glasses were randomized according to the cases that I kept them in.

Conclusion: This may be disappointing, but my impression was that they were all pretty similar. I was really unable to appreciate any major differences between any of them. If I were pushed to keep only one of them it would probably have been the Shamir Autograph II (but by a slim margin). A secondary conclusion, however, was that the freeforms were distinctly better than regular progressives such as Image, Concord, Accolade, Easy fit, Fairview, Solomax, Comfort and Outlook in almost all respects.

Disclaimer: It should be noted that personal experience with progressives is purely anecdotal and very dependent on the refractive and neuroadaptive status of the individual. My conclusions may therefore be entirely irrelevant to others:

[RT]

This is not quite the double blind test you think it was. A double blind test, in which neither the subjects nor the testers know the key aspects of the conditions being tested, is generally used to prevent bias or placebo effect. In this case, you knew something very important about the test...that all the lenses are categorized similarly.

Your first line indicates the inherent bias behind your test. "To sort out the hyped up claims..." (emphasis added). It's fairly clear that you had reached your conclusion that there wouldn't be any difference among the designs prior to starting the test. Going through the motions of the test just helps rationalize your long standing belief that wholesale acquisition price is the single most important determinant in choosing a lens design for your patients.

It's interesting that you did not decide to test fitting parameters as part of this test. Many Optiboarders have long noted that a great optician can fit a mediocre design and come up with good results for the patient. Conversely, poor fitting can nullify a great design. Barry Santini would assert that if he did your fitting, you'd walk away with the "Wow!" factor. What if the manner in which the designs were fit was actually the culprit behind the lack of difference between the designs? It may not be reasonable to assume that all progressives should be fit identically.

You should take your test to a 2nd level, and try different fitting techniques. If you found distinct differences in performance based on who did the fitting and how it was done, that would indicate a significant reason for people to use their local ECP rather than order online.

[Barry Santini]

The "wow" factor, RT, is a *comparative*;, i.e., what were they wearing (or not wearing) before.

For instance, EVERY single satisfied Comfort wearer I have LUVs and sees a difference with Physio Enhanced.

EVERY single SV wearer I have, with a cyl 0.75 or over, or a anomolous pupil height placement (or non-default POW values) sees a WOW with Auto II SV and/or Zeiss Individual.

Of course, if their corrected acuity or contrast sensitivity lies at 20/40 or worse binocularly, there is no benefit, IMHO.

FWIW

B

[ilanh]

This is not quite the double blind test you think it was. A double blind test, in which neither the subjects nor the testers know the key aspects of the conditions being tested, is generally used to prevent bias or placebo effect. In this case, you knew something very important about the test...that all the lenses are categorized similarly.

Your first line indicates the inherent bias behind your test. "To sort out the hyped up claims..." (emphasis added). It's fairly clear that you had reached your conclusion that there wouldn't be any difference among the designs prior to starting the test. Going through the motions of the test just helps rationalize your long standing belief that wholesale acquisition price is the single most important determinant in choosing a lens design for your patients.

It's interesting that you did not decide to test fitting parameters as part of this test. Many Optiboarders have long noted that a great optician can fit a mediocre design and come up with good results for the patient. Conversely, poor fitting can nullify a great design. Barry Santini would assert that if he did your fitting, you'd walk away with the "Wow!" factor. What if the manner in which the designs were fit was actually the culprit behind the lack of difference between the designs? It may not be reasonable to assume that all progressives should be fit identically.

You should take your test to a 2nd level, and try different fitting techniques. If you found distinct differences in performance based on who did the fitting and how it was done, that would indicate a significant reason for people to use their local ECP rather than order online.

I disagree most strenuously. The fact that the claims are "hyped" is indisuputable. No one on this forum (or anywhere else in the optical field) would dispute this fact. Your second point regarding my costs is also incorrect. My practice is 60% VSP and I pay nothing for the progressives that I dispense. It is the VSP lab that pays for it in full. Therefore, the costs of these lenses are irrelevant to me, I was interested only in their relative merits and which ones I should favor. In this sense, the trial is double blind since as the investigator I didn't really care which design "wins", and as the user I had no idea what I was using. Also, please note that this trial was not determined to compare freeforms with non-freeforms. Therefore, since all freeforms cost about the same (even in the non-VSP part of my practice), your point regarding "wholesale acquisition price" is simply not true. Your last point regarding different fitting techniques is probably the most erroneous of them all. I personally was involved in the fitting of these progressives. With all due respect to Barry Santini and his fitting prowess, I am no slouch either when it comes to fitting. Every frame in this study was preadjusted for fit. We then had the pupil dotted and a blue line drawn across the lenses. I then walked around for a few minutes with my head in various positions to determine that the blue line fell directly below my line of sight by 1-2mm. The blue line was adjusted if it did not. Seg heights were then compared in all cases and found to be within 1mm of expected. Regarding "taking my test to a 2nd level" If you've read my posts over the last few years you'll see that I devote an almost absurd amount of time to the art of fitting progressives. I've devoted as much time to this "test" as I want to. Take the results for what it's worth.

[optical24/7]

I agree with both Barry and the Dr. The Wow factor is relevant to what the patient is coming out of. My personal experience with newer designs along with trials I've conducted with our patient base has shown me that there are very small differences in modern PAL designs, only subtle nuances of difference.

One may be marginally better in intermediate distances, another slightly wider angles of distance acuity, another may have a slight advantage in near angle width, but none are drastically differing in feel or function. And higher prices do not indicate superiority.

Virtually all modern designs are superior to the PAL's of years gone by, but vastly different than each other today?...No.

[Barry Santini]

With all due respect to Barry Santini and his fitting prowess, I am no slouch either when it comes to fitting.

Ilanh:

This is the most repsect I've been accorded since I splayed open the internet cow. However, it is not appropriate. As I've said before, success with these lenses and the WOW is less about the fittting than what Optical24/7 & I know to be how we're changing their vision from what they're used to.

I would be glad to be fit (and refracted) by you...might even be able to see for once!

Thanks for all you post here. I learn alot from all the doctors who participate regularly.

B

[DocInChina]

Ilahn,

With all due respect, if you wanted to conduct a double blind study it would require that you have no knowledge about which progressives (brands, traditional surfaced and free form) that you were wearing. The frames should be identical in everyway which you mentioned you already did) and the lenses or frames should be labeled A, B, C directly on the frame. In the event the frames are put into the wrong case, you would still know which was Frame A, B, C, etc.

The better method of conducting this would be for you to choose a group of test subjects and record their sex, age, education and previous progressive wearing history and ask if they know which brand they are currently wearing. Actually there are more questions to add to this which would be important for statistical analysis.
The person taking the measurements should not be the same person to glaze the lenses. The person doing the dispensing should not be the person who glazed the lenses. If you then have 4-6 different lens brands/designs you would then create a test group of at least 30 people split between men and women. Then you would create a questionnaire asking the persons preference from best to worst and asking specific questions to ascertain why they preferred one set of glasses over the other. You could then create control group where all the glasses are one type of progressive but still giving them 4-6 frames labeled A, B, etc and asking them which performed best, etc, the same as you did for the non-control group.

For the purpose of analyzing test results, it would not be considered a double blind study if you are the tester and testee for a study no matter how impartial you considered yourself. The data would be considered contaminated.

We are currently conducting a double blind study (not ophthalmic related) and we had to go over all the testing details to insure the study would not be contaminated rendering the test results useless for our purposes.

Eric

[BMH]

I guess you should have picked a different phrase than "double blind", Huh?

The problem I am having in the office with some of these new digitally surfaced lenses boils down to the processing/surfacing. It seems it is ripe for inconsistant results.

It would be interesting to see a test like Ilanh's but this time the same digital lens and frame combination processed at different labs. I'd put money on it that they would not all wear the same. I realize that is not your point on this thread however.

I am glad you chose to share your findings, biased or not.

[DocInChina]

I guess you should have picked a different phrase than "double blind", Huh? .

I appreciate hearing Ilahn's opinion regarding his personal experience and I do know he takes prescibing and fitting progressives (as well as other designs) quite seriously. Opinions give us all feedback but opinions and standardized, formal testing do not necessarily yield the same results. My post was in no way meant to be an attack on Ilahn's post.

The problem I am having in the office with some of these new digitally surfaced lenses boils down to the processing/surfacing. It seems it is ripe for inconsistant results. It would be interesting to see a test like Ilanh's but this time the same digital lens and frame combination processed at different labs. I'd put money on it that they would not all wear the same.

Lab to lab differences could occur for several reasons. Most of those reasons would typically be due to management issues rather than equipment issues.

[bigeyejim]

Regardless of semantics, I think the findings are to the point, with the added experience of Barry and 24/7.

In my experience, what they are coming out of is very important to realise. We took a -5.75-1.25 OU patient out of regular 1.67 lenses and put him in Zeiss Individual FF SV. It is beautiful when someone puts on their specs and you actually hear them say, "wow." Or "Whoa, dude," depending on who is putting them on.

Same for progressives, but to a bigger extent. There are a lot of "oldies" for lack of better terms around my area. These suckers are still fitting Natural, Adaptar and "original" Kodak. Sure, you take them out of those lenses and they freak out.

Myself, personally, I wear both the Unique and the Individual. Is there a big difference? Ehhh, not really. The Individual IMHO is a better lens, with A LOT more clarity in all parts of the lens, whereas the Unique isn't quite as clear outside of the reading zone. They BOTH "brighten" things up outside, again, the "hi-def" experience. They are both better than my daily drivers which are GT2, also a great lens. I have worn Physio 360, and IMHO, this is no better than the GT2 - old school. (-1.00-1.25X092, -1.00-1.25X077, ++1.50)

Bottom line, if we are giving our patient the best, and they all deserve at least that opportunity, then they will be wow'ed by these new lenses.

[ilanh]

Ilahn,

With all due respect, if you wanted to conduct a double blind study it would require that you have no knowledge about which progressives (brands, traditional surfaced and free form) that you were wearing. The frames should be identical in everyway which you mentioned you already did) and the lenses or frames should be labeled A, B, C directly on the frame. In the event the frames are put into the wrong case, you would still know which was Frame A, B, C, etc.

The better method of conducting this would be for you to choose a group of test subjects and record their sex, age, education and previous progressive wearing history and ask if they know which brand they are currently wearing. Actually there are more questions to add to this which would be important for statistical analysis.
The person taking the measurements should not be the same person to glaze the lenses. The person doing the dispensing should not be the person who glazed the lenses. If you then have 4-6 different lens brands/designs you would then create a test group of at least 30 people split between men and women. Then you would create a questionnaire asking the persons preference from best to worst and asking specific questions to ascertain why they preferred one set of glasses over the other. You could then create control group where all the glasses are one type of progressive but still giving them 4-6 frames labeled A, B, etc and asking them which performed best, etc, the same as you did for the non-control group.

For the purpose of analyzing test results, it would not be considered a double blind study if you are the tester and testee for a study no matter how impartial you considered yourself. The data would be considered contaminated.

We are currently conducting a double blind study (not ophthalmic related) and we had to go over all the testing details to insure the study would not be contaminated rendering the test results useless for our purposes.

Eric

Eric,
Obviously you're correct in all you've said. My "double blind, randomized" trial is really just a personal series of observation and not by any stretch a real study. If I ever submitted this to one of my journals for peer review they would simply laugh. Incidentally, the study that you postulated above is one that several years ago I posted an open invitation on this forum for someone else in the optical field to do. The response that I received is that no lens company would want to grant funding for such a study because it would probably conclude that there were no real differences between their design and others. I now believe in this prediction. Of course, one huge detriment of this study is exactly what Barry and others have pointed out, which is it is very dependent on what Rx you're wearing, age, and what design you're coming out of. This means that to really capture the data you need sufficient myopes, hyperopes, astigmats, age groupings, lens design groupings etc. This easily could increase the numbers of participants to 200. I think that Dr. Sheety realized this in all his earlier studies and simply looked at Rotlex readings to arrive at his conclusions. I think that I and many others have probably arrived at the conclusion, even without such a study, that many of these designs are substantially identical in terms of general patient satisfaction.

[DocInChina]

ilanh,

Now that you mentioned it, I do recall the post you made several years ago. Thanks for the reminder.

I think that I and many others have probably arrived at the conclusion, even without such a study, that many of these designs are substantially identical in terms of general patient satisfaction.

Good point and I would be inclined to agree with you. This leads back to the purpose of a double blind study. Most practioners have their favorite brands/designs and likely telegraph their personal attitudes to their patients about any given product (as they should). What would be an interesting experiment is to subsitute the brand/design without informing the practioner and see if the reported success rate of the patients differed between the brands/designs.

Eric

Eric