What if Beta testing was your job?


#1

Got to thinking about general interest in the Beta program. What if you were asked to use your own Glowforge for Beta testing? Would you still be so quick to volunteer? Beta testing of the H/W is one thing but after the production units have begun to ship, it’s not as if the Glowforge capabilities will be cast in stone. New capabilities, many of them S/W related, will continue to evolve. It is likely that the company will continue to seek volunteers to provide user feedback for near term improvements. We might be asked to use our own production machines to provide that feedback. I’m not so much interested in whether this question is fair or not, because there would be wear and tear on your personal machine. This is really just an exercise in psychology. Whether those that volunteered did so because they have a mentality typical of makers or because they simply wanted a unit early. I for one was ready to volunteer even before the company explained the Beta units would be on their dime. How about you?


#2

I can’t say 0% of my desire for Beta was to get my hands on it early, but that was certainly a very nice bonus. I am an early adopter of any technology that sparks my interest and that by default makes you somewhat willing to troubleshoot and put up with things that are often not ironed out fully. As part of my job, while its not beta testing, I do R&D for software and hardware that I feel would benefit our product. I find that I truly enjoy the process and like the challenge of testing new things that have very small support structures in place.

I agree with you, that I was willing before knowing that we would get a replacement unit after the test period. I also was fully willing to test on all sorts of materials I have amassed and not just stick to the low cost or free.

That being said, I am also happy now to just wait and keep creating content to be used when it does get here. I also keep adding to my supplies as people point to new and interesting materials.


#3

I LOVE beta testing and revisionism. That is what my job has been for the majority of my life, whether its being a UX designer or a CTO. Finding problems, fixing them, finding more efficient ways of doing what needs to be done is what gets me excited. Theres nothing I love more. Its the reason I learned graphic design. Its the reason I learned to code. Its the reason I learned to make things in meatspace too.

There have been a number of projects that I have jumped onto and volunteered because I was a user, and wanted to see positive change in aesthetic and usability. I will happily use my glowforge to beta test new things if they give me the ability. Im planning on making my own upgrades anyways, might as well lump in the officially sanctioned ones too! Where do I sign up?!


#4

great statement/question! I’ve said before that even the production units are not limited to being our version 1.0. The way that the unit’s software is setup in a cloud based format means that (just like any other program) there are going to be periodic changes that some are going to like, and others are not. I might not have the technical skills to troubleshoot many of the laser’s issues on my own (which is why I volunteered on their dime :wink: ) but purchasers of production units are still testers to some degree. Isn’t the point of beta testing to work the majority of bugs out of a mechanism or system rather than make the unit perfect (impossible, I might add)? I did not expect a place in the beta tester line (those of us in Florida are the furthest away and least likely to recieve a beta unit) but my offer was there honestly and openly. and once the final production unit comes in and I gain a better understanding of laser capabilities I wont hesitate to offer suggestions for improvement (whether or not I am an official post-beta tester)


#5

I don’t think I would volunteer my own Glowforge as a beta unit/early model unless there was something like a commensurate gift certificate for products in the catalog, up to $2,000 off a final production machine.


#6

Understand that perfectly. That’s where my inability to feel the difference between $5 and $5,000,000 becomes a problem. The math is obvious but the emotions aren’t there. It’s just paper to me. Luckily my wife can translate it into lifestyle impact for us.


#7

As a software engineer, I find this an interesting question.
I would assume that Beta testing in our case would focus on Hardware (usability, function, form, etc.) and on the underlying framework and core features of the software. If I understand correctly, the software will be cloud based, leading me to believe there will be some sort of continuous integration/deployment/(pick you name for it) such that some specific features are too likely to change to be in scope for Beta.

Based on the above assumptions and my professional experience with testing IT products, I would do not think I would volunteer to be a Beta tester. I would feel compelled to do it right and be comprehensive (noting like an OCD tester!). As such, I’m sure it would take a LOT of time and involve doing a lot of things I’m not interested in (yet) or qualified to evaluate. E.g., I’m interested in working with leather, but no so much on “drawing on material and cutting it out” (which seems to be a selling point for a good sized portion of target market).

That being said, if I were just expected to Beta test based on MY desired usage profile and provide a somewhat thorough written report of my experience…eh, maybe I could go for that.

As for volunteering for Beta just to get my hands on a unit early? Not for me. I would be too tempted to play instead of work (and I think Beta testing is real work!).


#8

Not necessarily work. I give as an example… The Inventables company, maker of the XCarve CNC machines have regular updates to their user S/W. They allow people access to the Beta S/W on request to evaluate and comment back on the usability of the updates. The volunteers use their own machine to provide that feedback. Eventually the updates are released to the public. The downside is that there is wear and tear on the machine and sometimes very intricate and time consuming projects are ruined. Microsoft even sends out Beta S/W to volunteers long before we see the updates.


#9

I expect that I’ll be giving the company free feedback in terms of giving comments about functions and documentation, noting bugs, and making feature requests, so it’s kind of a moot point for me. I’d appreciate freebie materials, replacement filters, or somesuch as compensation for experiments they request, but I just love helping people develop things.


#10

I see your point. I was not thinking of Beta testing in those terms.

For an established product with a good size existing customer base, I can certainly see this kind of Beta testing prior to a general release of a new feature, etc. (or even to decide to make a release or not).

I guess I did not think about from that perspective because I was thinking of GF Beta testing as being ‘prior to initial release’ and involving a relatively smaller customer base (at least for now!).


#11

Interesting question. As I’ve been a beta tester from many software packages and operating systems over the years, this will be different. Sure, in the past I’ve used my computer to test (never felt I was really putting much at risk other than the likely requirement to reload everything) and provide feed back and debug information.

This will be different as the software will evolve in the cloud (assuming they offer any of us access to a beta cloud site) which would not impact my computer, but could potentially impact the Glowforge hardware itself if something goes wrong.

Assuming we are given access to a “beta” site, I expect I would be very willing to help test the next version on my personal hardware.


#12

That is exactly the kind of job I would love :slight_smile: Every job I have had I have transformed to focus on such aspects. But since the job itself isn’t really for doing that thing… I never feel like I have the experience required on my resume to chase that kind of a position :frowning:


#13

If you have a talent and a tendency, just go for it. Tweak your resume to accentuate those parts of the duties at those jobs. Problem solvers are my favorite type of people to hire.

In fact in most of the interviews ive done with candidates, I dont ask stock interview questions: where do you see yourself, whats your biggest fault etc. I ask questions that try to gauge how people will handle certain situations, and the steps they would take therein. I like to keep a number of strange scenarios on hand just to see how the candidate will approach it.

It was surprising how few people have good problem solving skills. Those are the type of people that are highly valued in places, regardless of whether they have a ton of experience or not, because those type of people tend to learn quickly and adapt. It never hurts to send out a resume to test the waters


#14

Doesn’t hurt. But requires some serious chutzpah. Either you are applying to a “big boy” business where people with lots of experience will be on par with you or even under you, or you are applying to a startup, where you if are lacking… they are dead :slight_smile:

Anyway, back on topic a bit, since I ignored the mean thread in my initial entry…

I don’t see it likely that the company does ask anyone to do as much. Offering Beta Access to software updates maybe… but other than that it seems unlikely they would have people in the field doing any special projects.

If they did have access to beta versions of the software, I would very likely make use of it. I like to have access to the fancy new features as soon as possible, and if something is going to break, I would rather break it while in a semi-malleable state. If my special use case is going to have issues with a new version of the software, I would prefer to figure it out and inform the company while they are still developing said problem causing update, rather than after it is released and I have no alternatives until a fix is devised and launched.


#15

I’d do it. I have no issues with dealing with the ‘pain’ so standard end users don’t have to.
I am currently in a bunch of software/hardware betas right now. I personally find it fun helping/finding issues with pre-released products. I am guess I’m weird that way.


#16

I think I would.
Once my 'Forge arrives, I plan to contribute to the forums if I have anything at all to add. So many of you have such experience and ideas that I don’t always think my questions or observations will add to the discourse; but I certainly would like to.
I read reviews before buying goods and services, and when the reviews are not sufficient, I provide what information I can to make them sufficient.
If I’m fixing on staying in a cheap hotel and the entire world wide web only says, “breakfast included,” then I figure I better define whether that’s a pile of 3-day-old lemon danishes and coffee, or whether there’s going to be eggs and bacon.
When YouTube can’t tell me how to fix a broken part on my 13-year old minivan, I figure it out, take the videos, and sometimes even get around to posting.
If the information is there, no point repeating it. A beta, though, would give all kinds of opportunities to add the the knowledgebase - both for the company and for all you other users.
It’s fun to be a part of something. It’s satisfying and exciting to help build something, especially something new that’s going to disrupt. I figure that’s the biggest thing the beta testers get out of it.


#17

The beta test being proposed is a s/w upgrade.

The original beta request was for 10 hours a week which included designing, actual use, and writing up the results. As actual time running the laser would be relatively minimal, I would be okay with the decrease in my laser tube’s remaining life. There is also a risk the s/w changes could send values to the drive motors that somehow damage them or drives the laser head into something that damages it. If the damage is severe (as in noticeable) my guess and expectation is that they would replace the parts or the unit. If the damage was subtle it is probably slipping through the beta test period and I’m going to fall prey to it anyways.

I would expect that to be offered this beta opportunity I would be doing work that would qualify me for really wanting this feature. If you didn’t already do work that would exercise at least one of the new features/improvements you’d only slow testing down. Therefore, I’d probably have designs and materials more or less ready to go.

So assuming I wasn’t underwater with the rest of my life, yes I would volunteer. I see the risks as minimal and I would want to be able to influence features/improvements that will directly impact the work I do with the glowforge. If for some reason the glowforge staff were confused about what I do with my laser, and essentially offer me the opportunity in error, I’d decline. I have too much to do to waste their time and mine evaluating features I won’t use, will only use infrequently or don’t know I will use a lot in the future.


#18

Hi I’m new to responding to the forum but I read it regularly. I have a tendency to unofficially beta test all my tools when I get them so I would volunteer. I have a few other CNC machines and am always testing materials that can be cut, how fast and how deep I can make passes to maximize their capabilities. With a laser it seems safer then other tools I own like my CNC mill and CNC router because there is no physical contact with the material. (Less shattering bits). In a way when I finally do get a glowforge
I will be testing it anyway. I appreciate all the comments people make because it helps give me boundaries when I do start using it.


#19

No laser experience so I couldn’t beta the hardware.

As far as the software goes, I might volunteer to beta test that if I had nothing going on at the time. Wouldn’t matter if I was using my machine - I’d probably do it for a decent supply of materials to play with, or replacement filters or something.

What i really don’t like about software testing, is that every fix that gets applied seems to break a half dozen other things. And each iteration has to be checked completely from top to bottom to find all of the problems that the last fix created.

Best thing to do in a software testing situation is create a checklist of everything that needs to be functional in the release, and just check off the list as you test each function. Each and every time. (Tends to be tedious.) But it’s the quickest way to get to full functionality, 'cause otherwise, you’ll forget whether you tested things, and you can get several updates in the space of a few days as things get fixed, then the problems from the first fix get fixed, and on and on and on. What i did was check for the initial fix that we’d asked for, then immediately start over at the beginning of the list and work through everything else to check it again. Problems got marked and a fresh list started. Lather, rinse, repeat.

Once you have completely run through the entire list successfully - it’s ready to release, and not before. The worst thing in the world is a broad distribution of buggy software that you didn’t test completely and think is fine. (The screaming can shatter ear drums. :scream:)

And even if the software will be maintained in one place on the cloud, it still has to be checked completely. (It’s just easier (and smarter i think) to fix the problems on one copy than float out continual updates for months after any major upgrade.)

Software testing is a lot of work that you wouldn’t expect - it might change some folks minds to know how much. Just throwing that out there so everyone can clearly see the hungry gator in the pool before jumping in with both feet. :open_mouth:


#20

Luckily S/W tools have come a long way. With these tools automatic testing of functions and user interfaces is now the norm for complex S/W. It would be almost impossible to provide timely updates otherwise. These tools lessen considerably, but do not totally eliminate, the need for manual test procedures you describe.