Interesting Pages from MicroSoft Tester Center

I've dug out a few really interesting pages from Microsoft Tester center [http://msdn.microsoft.com/en-us/testing/default.aspx] space hosting on MSDN.

The couple technical-oriented links talking about some Code/Design review aspects and practices at Microsoft:

http://msdn.microsoft.com/en-us/library/ff519670%28v=MSDN.10%29.aspx

http://msdn.microsoft.com/en-us/library/ff521647%28v=MSDN.10%29.aspx

The following 3 links from on management and improving processes from a personal blog of Microsoft Test manager
http://blogs.msdn.com/anitag/archive/2009/03/22/business-focused-vs-people-focused-managers.aspx

http://blogs.msdn.com/anitag/archive/2009/03/29/automation-maturity-what-should-a-test-manager-focus-on.aspx

http://blogs.msdn.com/anitag/archive/2009/08/19/should-bvts-pass-100.aspx

I don't think that MS Tester center is super resource with always fresh and novice content. I believe it's a bit static and archaic but for someone it can be good start point in testing journey. More attractive and concrete Microsoft!

Hybrid-level test automation is the most effective

Hey test automation gurus and those who just in the beginning of this challengeable road!

After reading http://googletesting.blogspot.com/2010/04/googles-innovation-factory-and-how.html I’ve got some thoughts about test automation effectiveness from test-level point of view. Most of developed s/w products have various interfaces lying within different layers. Pulling these interfaces in tests we get some testing level coverage. Just for example, working through GUI, test likely to be called full System E2E level (aka Integration), going down we can call objects of GUI shell representing native methods and properties of GUI entities (calling Get/Post requests, working through DOM, COM objects). The last approach is still integration testing or system level but bits farther from User’s use case – it means we don’t test GUI itself.

Component (module)-level functional testing is deeper one as tests here operate over a particular component using its native external interfaces. Let’s call it API-level testing. This approach requires being subscriber and maintainer of module API. Surely this testing is quicker and more robust at most cases due to absences of intermediate layers and integration containments.

Components integration potentially can utilize different levels of interfaces but this is drawback of supporting different technologies. Here is example: run system job through Component A API, validate Component B response through GUI. The better case: run system job through Component A API, validate Component B in database. The most valid case for this testing level: run system job through Component A API, validate Component B behavior using Component B API. The ideal case: Components A and B have single unified API or have higher layer inheriting and unifying all components API’s and externalizing secured API.

The Unit-test level is when you test product’s code itself (Classes, methods and structures). This is important level which must be used for producing good apps. Unit testing to be relied on development resources.

Hybrid-level model in essence is a mix of all (though probably but Unit-level). But the formula of proportions and efficacy is different from project to project. I just can show some examples of reasonable scenarios of combining different test levels:

• API level testing is using to manipulate system components, transmit data and trigger events. Verification is running on API-level, then in GUI as double checkpoint.

• Integration between 2 components is verifying in module integration level (API). Integration with GUI layer is checking in GUI as checkpoints.

• Change data in database (SQL scripting); checking results in API as checkpoint. And vice versa – change data using UI, validate on database side (SQL querying). Let’s complicate these cases by adding validation in intermediate layer (e.g. check Web service I/O, check queue object, check POST/GET Header and Body)

Google Testing Blog: Google's Innovation Factory (and how testing adapts)

This is amazing that GoooGle reveal their some internals about testing, continious inegration, ROIs and savings of different approaches and digits. Really appreciated! See theirs presentation here:

Weekly Fun Relax on FR : QABurger

What are differences between professional and matured QA and so-called QA as some organizations like to use this term but don't follow proper pracices? The answer is simple - they differ like different fast food and restaraunt meal cooked by skilled cooker.

So to cook QABurger, you need to mix:
- developers response for testing - so they combine development and testing;
- practice Ad-hoc testing only without Test plans and schedules;
- gather all issues and defects somewhere in any format - just write something to revisit one day;
- don't prioritize and plan fixes and new features - let's to begin from easy task, then go down by fix/implementation complexity;
- use very simple and clear metric of you quality attribute like number of defects - that's enough to make some judgements and to define scope of works;
- simplify all processes - chaos is a pat of creativity;
- deploy new features and fixes on production as soon as you can after checking on development machine - more frequent commits simplify and accelerate work
- give your system for testing for potential Users - testing engineers will yield the same result but for money

Data Migration/Transformation testing tips

Data migration/transformation projects are actual in s/w industry especially because systems mature, systems integration, needs in transforming data from legacy systems, for data improvement, consolidation, separation and purifying tasks. Some data migration activities may be so complex that require involvemnet of significant resources, skilled professionals and special tools. It can be truly noticed that data transformation itself in modern projects is sophisticated and challengable solution requiring individual approach and process modeling.

Data migration testing runs as grey or even white-box testing at most cases. Simple verification against on presentatoin layer can't gurantee good coverage, discovering hidden defects and this testing itself is not effective as requiring intensive data verification without deep inside research.

Here are my tips for testers who run or are going to test data migration:

- learn purpose and business model of project and Company to realize migration target

- study transformation schemas and scripts(XSD, SQL scripts)

- study development algorithms which operate over original data to produce transformed one. Notice you have to correspond the algorithms with specifications and requirements on migration process

- study database schemas (all from original to final through intermidiate states)

- develop and check your own scripts (SQL, XML, XLS, XSLT) to retrieve data for testing

- design sufficient and efficient data sets for testing data validity and integrity

- store your scripts and data sets under version control and link with versions of AUT

- run tests and store actual outputs in test logs.

- Also consider saving test logs as database records in testing DB with linking on defect entries. This approach will allow to revisit test run results as soon as you need in structured view

- Keep your knowledge about everything in product changes up to date

- Check data transformation in all points where it's being transformed in a whole product (DB, presentation, services, batch jobs, intermidiate mediums, etc)

- Make sure and double check correspondence and compatibility of original data types and transformed types for each entity, field and macro-data. This is a part of getting familiar with database architecture and schemas

- Run test frequenly against various data sets representing various combinations and tricky moments (like using special characters, sontrol symbols, code snippets, too long strings, invalid types and mixed content; strings that potentially can be truncated, encoded or echoed)

- More CREATIVITY and Hacking! You should BREAK IT!

For those who need to continue learning this topic, take a look at Wikipedia's related pages and this article about Data warehouse testing

http://www.developeriq.in/articles/2010/mar/10/data-warehouse-testing/