Automation - Releasing software continuously with consistent quality
There are a lot of challenges in building great software, many technical leaders have written many great books about it. I like to read them, and I have learned a lot from those great minds. Understanding it theoretically is one thing, applying it, and be successful at doing it is different.
Releasing frequent software improvements with consistent quality has been the nearest and dearest to my heart. I have been burned badly many times by software releases that are supposed to improve the product, however, break critical features used to work, infuriated customers. To be honest, who can, honest to the god, say that they have never been burned by this! It doesn't matter whether you want to face it: there is no bug free software. Software are developed by human, tested by human too. Human make mistakes, regardless how great he/she is.
If that is the case, should we give up? I used to work with one large brand name customer at my previous job, who categorizes software bugs as "fact of life"... I don't believe it. Yes, it is impossible to produce bug free software, however, it is possible not to make the same mistake again, not to break things used to work. And I think write program to test your program is the key.
Predictability and consistency are the fundamental reasons that I choose computer science as my career. Call me "control-freak", however, I like things to be predictable, I like stuff happening for a reason. Who doesn't like it! Life is chaotic already, who could have predicted the high flying Bear Stearns stock used to worth $170 one day and turned into $2 the other day!
Human is unpredictable, that is why people management is hard, whereas computer is predictable and consistent. If it gets one thing right once, you know it will always get it right, regardless how many times you ask for it. We all know that successfully releasing software once is not hard, the hard part is to continuously releasing software successfully. To do this successfully we have to rely on computer, not human.
In a note to my team today, I emphasized the importance of QA automation. I am very happy to see that they took this to their hearts too. Both core and UI teams have setup automated continuous build and regression test framework. Post V1, our biggest challenge is to do monthly releases continuously and consistently. Automation is going to help.
Disclaimer: I am not down playing the capability of human, after all, all programs are written, tested by human
Releasing frequent software improvements with consistent quality has been the nearest and dearest to my heart. I have been burned badly many times by software releases that are supposed to improve the product, however, break critical features used to work, infuriated customers. To be honest, who can, honest to the god, say that they have never been burned by this! It doesn't matter whether you want to face it: there is no bug free software. Software are developed by human, tested by human too. Human make mistakes, regardless how great he/she is.
If that is the case, should we give up? I used to work with one large brand name customer at my previous job, who categorizes software bugs as "fact of life"... I don't believe it. Yes, it is impossible to produce bug free software, however, it is possible not to make the same mistake again, not to break things used to work. And I think write program to test your program is the key.
Predictability and consistency are the fundamental reasons that I choose computer science as my career. Call me "control-freak", however, I like things to be predictable, I like stuff happening for a reason. Who doesn't like it! Life is chaotic already, who could have predicted the high flying Bear Stearns stock used to worth $170 one day and turned into $2 the other day!
Human is unpredictable, that is why people management is hard, whereas computer is predictable and consistent. If it gets one thing right once, you know it will always get it right, regardless how many times you ask for it. We all know that successfully releasing software once is not hard, the hard part is to continuously releasing software successfully. To do this successfully we have to rely on computer, not human.
In a note to my team today, I emphasized the importance of QA automation. I am very happy to see that they took this to their hearts too. Both core and UI teams have setup automated continuous build and regression test framework. Post V1, our biggest challenge is to do monthly releases continuously and consistently. Automation is going to help.
Disclaimer: I am not down playing the capability of human, after all, all programs are written, tested by human
自动化测试当然非常重要，若能做到这一步将非常理想。可惜的是，国内大多数IT公司没有真正意识到这个问题、也很少有能力去做到自动化测试。
没能力，没意愿，业务变化飞快，你让他们怎么能去考虑自动化测试呢?
看到bug free software很亲切，因为我做了一款自由软件"借鉴微软公司软件研发理念的自由软件：缺陷管理系统BugFree 2.0":
http://www.bugfree.org.cn/bugfree2/Doc/AboutBugFree.txt
若贵公司真的做到了自动化测试，那么我相信BugFree将非常适合你们的测试管理，因为BugFree现在把Test Case, Test Result和Bug有机的管理起来了。呵呵，也算是一个广告吧。
Reply to this
别听那个bugFree的广告，一听“微软公司软件研发理念”就知道是骗人的。自动化测试完全是人的问题，跟什么管理工具无关。我在知名外企实施过 TDD 开发，结果无疾而终。FreeWheel相比要成功得多。
Reply to this
多谢捧场
Regarding test automation, I say not only companies in China doesn't realize it's importance, you don't easily run into a company in US pay enough attention to this either.
However, I think the more frequent your software releases are, the more important test automation is. In FreeWheel's case, we are doing monthly releases, if we don't invest efforts in automating regression test, we would need an army of QA engineers to manually test the entire system every month. First of all, it takes a long time to complete regression test manually, secondly, once your regression test cases exceed certain number, there will be no human can remember all the test cases and repeat the testing steps consistently every month. You would be burned more frequently by the bugs in production.
I will check out your link!
Reply to this
To dongbin网友： 同意自动化测试是人的问题，但“不完全是”。你总得管理Test Case,跑一遍Case总得记录Test Result吧，Result是Fail的时候，你得记录一个Bug吧？
没有好的工具支撑，人是可以做记录，但会非常累。
我不知道为何“一听就是骗人的”，为什么要骗你呢？这个BugFree确确实实是“Clone”微软内部的Product Studio。这里你可以上去实际体验一把：
http://www.bugfree.org.cn/demo/
（不愿注册的话，可以缺省的用户名密码都是 demo)
现在北京有很多外包工程师给微软做测试，你可以找一位外包或你认识的微软工程师，看看BugFree是不是和他们现在用的Product Studio非常的相像。
关于微软的软件研发理念，有空的话请快速读一下：
Bug管理的经验和实践(http://www.liuzhenfei.com/zhenfei/resume/Document/BugFree.htm)
如果读完后，还是觉得我在骗人，那我会保持沉默不再说话了。
Reply to this
Great post i have ever saw about test automation, i have fully experienced the test automation build from bottom. it's not only limit to cost, quality, you should treat it as a project from the begin and evaluation the ROI. looking to communicate more with you.
Reply to this
I had honor to know you at the Hero conference. You are the type of leader I appreciate, and i will i will work for you someday.
Reply to this
agree. you are the star at hero conference, I hope to join your team too.
Reply to this
I am sorry.The last sentence should be i wish i will....
Reply to this
QA is important. A good architecture can make QA's life much easier.
Reply to this
刘振飞 是个值得敬佩的家伙。
Reply to this