<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Stefan's virtual Home</title>
	<atom:link href="http://www.stefanbuehlmann.com/s/?feed=rss2" rel="self" type="application/rss+xml" />
	<link>http://www.stefanbuehlmann.com/s</link>
	<description></description>
	<lastBuildDate>Sat, 02 Apr 2011 07:35:46 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.1.2</generator>
		<item>
		<title>Tester</title>
		<link>http://www.stefanbuehlmann.com/s/?p=186</link>
		<comments>http://www.stefanbuehlmann.com/s/?p=186#comments</comments>
		<pubDate>Sat, 28 Mar 2009 17:30:38 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[Tester regression test execute batch program file database content compare actual expected data ignore variable load unload template]]></category>

		<guid isPermaLink="false">http://localhost/s/?p=186</guid>
		<description><![CDATA[<p> My Favorite Error&#8230; <p>We once got the following error, while testing an important program of our customer. For the ones not fluent in german, it says: &#8220;REP-0999: Not implemented error&#8221;. About a year later, when we tested the next release, the error was implemented, imagine!</p> <p></p> <p>I was a project leader and software [...]]]></description>
			<content:encoded><![CDATA[<p><a name="Top"></a><br />
<h2>My Favorite Error&#8230;</h2>
<p>We once got the following error, while testing an important program of our customer. For the ones not fluent in german, it says: &#8220;REP-0999: Not implemented error&#8221;. About a year later, when we tested the next release, the error was implemented, imagine!</p>
<p><img style="width: 304px; height: 131px;" alt="my favorite error message" title="my favorite error message" src="wp-content/gallery/varia/MyFavoriteError.jpg"></p>
<p>I was a project leader and software architect in many software projects. I&#8217; ve seen the same errors again and again, I have seen simmilar solutions again and again and I still test&#8230; Over the years, I developped some very simple test approaches, which usually cover 90% of my testing needs. The concepts evolved over many years. </p>
<p>To support this, I wrote a little program called &#8216;Tester&#8217;, which summarizes all this experiences. The program streamlines the ideas I collected over the years. You&#8217;ll find it under <a href="#Download">Download</a>. I am happy to share it with you for free.</p>
<p>Tester allows to execute a sequence of test steps, where each step is a batch program (input/output oriented program without GUI). After each step, actual file output and actual database content can be compared against expected data, while ignoring the parts, which are known to be variable. Tester is ideal for regression tests. Tester can load and unload databases. Tester can also be used without database.</p>
<p></a><br />
<h2>User Manual</h2>
<h3>Contents</h3>
<p><a href="#ChapIntroduction">Introduction</a><br />
<a href="#ChapSeqOfSteps">Sequences of Test Steps</a><br />
<a href="#ChapStructOfSteps">Structure of a Test Step</a><br />
<a href="#ChapFilehandling">The Handling of File Output</a><br />
<a href="#ChapDBhandling">The Handling of Database Content</a><br />
<a href="#ChapVolumePerformance">Data Volume and Performance</a><br />
<a href="#ChapExecuteATest">Execution of a Test</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#ChapCallStack">The Call Stack</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#ChapCallingTester">Calling Tester</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#ChapProperties">Properties / Parameters / indirection Feature</a><br />
<a href="#ChapBillingExample">Billing Example</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#ChapBillingExampleDM">The Data Model of Billing Example</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#ChapBillingExampleRunning">Running Billing Example</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#ChapBillingExampleFeatures">Features demonstrated by Billing Example</a><br />
<a href="#ChapFAQs">FAQs</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ_Downloader">How do I dump the contents of a database?</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ_Uploader">How do I upload data into a database with Tester?</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ_StartAnewSeq">How do I best start creating a new sequence of test steps?</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ_DefineExpected">How do I best create the expected results?</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ_VersionControl">Which files should go under version control?</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;<a href="#FAQ_SecondStep">Why does the second step only contain expected_data_full but no Step.bat?</a><br />
<a href="#Download">Download Tester</a><br />
<a href="#Requirements">Requirements</a><br />
<a href="#Databases">Tested Databases</a><br />
<a href="#Installation">Installation</a></p>
<h2><a name="ChapIntroduction">Introduction</a></h2>
<p>Tester is a Tool to automate Testing. The highlights are:</p>
<ul>
<li>Black Box Test:Tester can execute a black box tests of a batch program (input/output oriented program without GUI). Tester feeds input to the program and compares its output against expected output, while ignoring the parts, which are known to be variable (timestamps, sequence numbers, installation paths&#8230;)</li>
<li>Grey Box Test:In the case of database-based programs, Tester can additionally verify the state of the database at the end of each step. Since this is a verification of an internal state, it is called a grey box test in this document.</li>
<li>Test sequences: Tester can execute single tests, entire test sequences or subsequences thereof. In the case of database programs, Tester first loads the database with the correct initial state. This allows decoupling development activities.</li>
<li>Regression tests: Tester can execute Tests again and again, and verifies that the results remain the same each time (but ignoring the variable parts).</li>
</ul>
<p>The following picture shows the structure a test sequence (<a href="#ChapBillingExample">Billing Example</a>) and of one of it&#8217;s test steps (0002).</p>
<p>A test sequence, here directory C:\Temp\Tester\Billing_Example, consists of any number of subdirectories, the test steps. Here the test steps are 0000, 0001, 0002 and 0003, but they might have other names, as long as they sort alphabetically in the order the test steps should be executed, see <a href="#ChapSeqOfSteps">Sequences of Test Steps</a>.</p>
<p>Each test step is a subdirectory, which has the following structure. Most important is the file <a href="#Step.bat">Step.bat</a>, which calls your application under test. Then the test step contains up to five subdirectories (<a href="#actual_data_full">actual_data_full</a>, <a href="#actual_data_masked">actual_data_masked</a>, <a href="#expected_data_full">expected_data_full</a>, <a href="#expected_data_masked">expected_data_masked</a> &amp; <a href="#data_definition">data_definition</a>) and up to four log files (<a href="#actual_full.log">actual_full.log</a>, <a href="#actual_masked.log">actual_masked.log</a>, <a href="#expected_full.log">expected_full.log</a> &amp; <a href="#expected_masked.log">expected_masked.log</a>). Further explanations about the structure can be found in chapter <a href="#ChapStructOfSteps">Structure of a Test Step</a>.</p>
<p><img style="width: 486px; height: 446px;" alt="Directory Structure" title="Directory Structure" src="wp-content/gallery/varia/DirStructure.jpg"></p>
<p>The following picture shows the concept of Tester. Within a test step the batch program called <a href="#Step.bat">Step.bat</a> calls the application under test. It takes (test-) input and produces (test-) output. The latter is compared against a predefined expected log.</p>
<p>And since the tested program might modify a database, at the end of the test step, the content of this database can be dumped and compared against expected data.</p>
<p>If both comparisons (log file and database dump) succeed, the test step is considered successful.</p>
<p>The concept allows to start testing at any of the test steps. If testing should start at step x, Tester empties and then loads the database with the (expected) content of step x-1 before step x is executed. Loading of data only happens before the first step of a sequence. See the two arrows labeled &#8216;continue or load expected_data&#8217;. The details about the sequencing of test steps you find under <a href="#ChapSeqOfSteps">Sequences of Test Steps</a>.</p>
<p><img style="width: 468px; height: 370px;" alt="Overview of Test Steps" title="Overview of Test Steps" src="wp-content/gallery/varia/Overview.jpg"></p>
<p>To get to know Tester in more detail, it is a good idea to go on as follows:</p>
<ul>
<li>Verify, that all <a href="#Requirements">Requirements</a> are met.</li>
<li>Follow the <a href="#Installation">Installation</a> instructions.</li>
<li>Then run <a href="#ChapBillingExample">Billing Example</a>, which covers most features of Tester.</li>
<li>Then read the following chapters.</li>
</ul>
<h2><a name="ChapSeqOfSteps">Sequences of Test Steps</a></h2>
<p>A test sequence is a directory consisting of any number of subdirectories, the test steps. Often the test steps are named 0000, 0001, 0002 and 0003, but they might have other names, as long as they sort alphabetically in the order they should be executed. The ordering defined by this method depends upon the underlying system. On UNIX systems, alphabetic case is significant in comparing pathnames; on Microsoft Windows systems it is not. There might exist a subdirectory named <a href="#defaultsDir">defaults</a>, which has a special meaning and therefore is not considered to be a test step.</p>
<p>Tester is started with the test sequence directory as its working directory and loops over its subdirectories, which are the test steps. The steps are processed in alphabetical order. Since each test step might leave the system in a specific state, the order of their execution is significant.</p>
<p>Tester can be used with or without database, see <a href="#tester.nodb">tester.nodb</a>. But if a database is used, the first two steps are special:</p>
<ul>
<li><a name="FirstStep">First Step</a>: The first step (often called 0000) must create an empty database (dropping and recreating the tables).</li>
<li><a name="SecondStep">Second Step</a>:The second step (often called 0001) must fill it with initial data.</li>
</ul>
<p>It is possible to process only a given range of test steps, by defining the properties <a href="#Prop.from">tester.from</a> &amp; <a href="#Prop.to">tester.to</a>. To make results reproducible, Tester recreates the database (if one is used) as it would have existed immediately before <a href="#Prop.from">tester.from</a>. It calls the first step, to recreate an empty database and then it fills the database with the content as expected at the end of the step immediately before <a href="#Prop.from">tester.from</a>.</p>
<p>Step.bat can stop the execution of the sequence and subsequent steps by returning exit-code 99 or 98. 99 causes an erronous halt, while 98 causes a halt without error.</p>
<p>If the database is in a known state, it is also possible to continue, without recreating the database fist. Use property <a href="#Prop.noload">tester.noload</a> for this.</p>
<h2><a name="ChapStructOfSteps">Structure of a Test Step</a></h2>
<p>The subdirectory corresponding to a test step contains the following (please refer to the screenshot above):</p>
<ul style="margin-top: 0cm;" type="disc">
<li><a name="Step.bat">Step.bat</a>: Step.bat is a batch script executing the code to be tested for that test step. Step is the corresponding Linux bash-Script. The following documentation only refers to Step.bat and implicitly means the bash-Script in the case of a Linux environment. Step.bat can call any other (batch) program. The standard output of Step.bat and the called program is captured and compared against expected output. At the end of the step the database content can be compared, see the following two subchapters. Take Billing_Example/0000/Step.bat as a template. Step.bat is optional. It can be omitted, if no actual test activity should happen on a given step and only loading or unloading of the database is required. <a href="#environmentvariables">Environment Variables</a> defined when Tester is called, are passed on to Step.bat. Step.bat can stop the execution of the sequence and subsequent steps by returning exit-code 99 or 98. 99 causes an erronous halt, while 98 causes a halt without error.</li>
<li><a name="data_definition">data_definition</a> &amp; <a name="defaultsDir">defaults</a>: data_definition is a subdirectory containing definitions about what to ignore when comparing file output (<a href="#log_masks.csv">log_masks.csv</a>) and database content (<a href="#tables_masks">tables_masks.csv</a>) for that step. data_definition is mandatory. Remark, for both files (<a href="#log_masks.csv">log_masks.csv</a> &amp; <a href="#tables_masks">tables_masks.csv</a>) there exist defaults in the directory <a href="#defaultsDir">defaults</a>, which is on the same level as the test step subdirectories. data_definition might also contain the marker file <a href="#listed_tables_only.txt">listed_tables_only.txt</a>.</li>
<li><a name="actual_full.log">actual_full.log</a>, <a name="actual_masked.log">actual_masked.log</a>, <a name="expected_full.log">expected_full.log</a> &amp; <a name="expected_masked.log">expected_masked.log</a>: Within the test step directory, these files contain the output of <a href="#Step.bat">Step.bat</a> (stdin, stdout, stderr of your program under test) of the given test step. &#8220;expected&#8221; stands for the log output expected at this test step and &#8220;actual&#8221; stands for the output it actually produces. &#8220;full&#8221; means the log &#8220;as is&#8221; and &#8220;masked&#8221; means regular expressions have been applied and strings expected to be differing are removed. Please also refer to <a href="#ChapFilehandling">The Handling of File Output</a> &amp; <a href="#FAQ_DefineExpected">How do I best create the expected results?</a>.</li>
<li><a name="actual_data_full">actual_data_full</a>, <a name="actual_data_masked">actual_data_masked</a>, <a name="expected_data_full">expected_data_full</a> &amp; <a name="expected_data_masked">expected_data_masked</a>: These test step subdirectories are only needed, if a database is used (relevant for detection by Tester are expected_data_masked and <a href="#tester.nodb">tester.nodb</a>). The directories contain the database content in four variants. &#8220;expected&#8221; stands for the state the database should have after the step and &#8220;actual&#8221; stands for the content it actually has. &#8220;full&#8221; means all tables with all columns and &#8220;masked&#8221; means only tables and columns under test (masked columns excluded). If the contents of actual_data_masked vs. expected_data_masked differ, the test step is said to have failed. Please also refer to <a href="#ChapDBhandling">The Handling of Database Content</a> &amp; <a href="#FAQ_DefineExpected">How do I best create the expected results?</a>.The existence of expected_data_masked tells Tester, that the database content should be compared, and causes Tester to fill the two actual directories. Each time the program is run, actual_data_masked is recreated by processing actual_data_full, since actual_data_full could change each time. But remark, it would not be efficient to recreate expected_data_masked from expected_data_full each time, because expected_data_full is meant to be stable. So it&#8217;s the responsibility of the person defining the test steps, to maintain this redundancy. But this is usually very simple, since when you have correct records in the actual directories, you can use these records for the corresponding files in the expected directories. It might be a future enhancement to introduce a switch, asking Tester to derive expected_data_masked from expected_data_full. expected_data_full is the master and expected_data_masked should always be in sync with it, because it is required to fill the database in the case processing should start at the subsequent step.</li>
<li>ORACLE_expected_full.log, DB2_expected_full.log &amp; MYSQL_expected_full.log (%<a href="#DB_TYPE">DB_TYPE</a>%_expected_full.log): If these files exist, the file corresponding to the chosen database is copied over expected_full.log before the test sequence is executed. This allows having different expected logs per database. If this multi-database test feature is used, special care for this redundancy is required. If one database runs fine, it is quite acceptable to declare actual logs to be expected, if the contents of the .csv files are equal.</li>
<li>Within a test step there might exist other subdirectories or files belonging to <a href="#Step.bat">Step.bat</a> or the application under test.</li>
</ul>
<h2><a name="ChapFilehandling">The Handling of File Output</a></h2>
<p>The script <a href="#Step.bat">Step.bat</a> calls the application under test. Tester captures the standard input (stdin) and outputs (stdout &amp; stderr) of <a href="#Step.bat">Step.bat</a> to file <a href="#actual_full.log">actual_full.log</a> in the test step directory. <a href="#Step.bat">Step.bat</a> should redirect or copy all output of the programs it calls to either standard output or standard error. At least this should be done for all output files which are to be verified. Sometimes it might be helpful to do the same with the (test-) input. In <a href="#actual_full.log">actual_full.log</a> data of standard input (stdin) is prepended with &#8220;INP&gt;&#8221;, data of standard output (stdout) with &#8220;OUT&gt;&#8221; and data of standard error (stderr) is preceded with &#8220;ERR&gt;&#8221;, see <a href="#Linetypes">Line types</a>.</p>
<p><a name="RegularExpressions"> Regular Expressions</a>, <a name="log_defaults.csv">log_defaults.csv</a> &amp; <a name="log_masks.csv">log_masks.csv</a>: After each step, Tester compares &#8220;actual&#8221; against &#8220;expected&#8221; logs. This allows making sure, program output remains the same across multiple invocations. But <a href="#actual_full.log">actual_full.log</a> &amp; <a href="#expected_full.log">expected_full.log</a> might contain changing strings which should not be taken into consideration when comparing the logs, e.g. timestamps, IP-addresses, usernames, passwords, pathnames and the like. Tester applies regular expressions to mask out such differences. The regular expressions are defined in ../<a href="#defaultsDir">defaults</a>/<a href="#log_defaults.csv">log_defaults.csv</a>, which applies to all steps, and <a href="#data_definition">data_definition</a>/<a href="#log_masks.csv">log_masks.csv</a>, which applies to the current test step only.</p>
<p>Applying regular expressions to <a href="#actual_full.log">actual_full.log</a> yields <a href="#actual_masked.log">actual_masked.log</a> and applying them to <a href="#expected_full.log">expected_full.log</a> yields <a href="#expected_masked.log">expected_masked.log</a>.</p>
<p>The regular expressions are the ones of Java 1.4. They are documented <a href="http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html">here</a>. Recall, that the regular expressions in the .csv-files must be correctly escaped. To be read correctly, a single &#8220;\&#8221;-character must be doubled. Second, since the &#8220;\&#8221;-character is also the escape-character of regular expressions, you have to use a sequence of four &#8220;\&#8221;-characters, if it should not be interpreted by the regular expressions engine.</p>
<p>The special Token SKIP used in the replacement part of a matching expression causes the entire line beeing removed from the <a href="#actual_masked.log">actual_masked.log</a> and <a href="#expected_masked.log">expected_masked.log</a> files. This comes handy, if the number of output lines of the program under test varies and should be ignored.</p>
<p>Examples:</p>
<ul>
<li><a href="#RegularExpressionsExample">Regular Expressions in Billing Example</a> is a good example for newbies.</li>
<li>The following regular expression removes trailing zeroes behind the decimal point. Put it into file <a href="#defaultsDir">defaults</a>/<a href="#log_defaults.csv">log_defaults.csv</a> to try it out.<br />
<code>"(,)([0-9]*)([1-9])(0*)","$1$2$3" </code></li>
</ul>
<h2><a name="ChapDBhandling">The Handling of Database Content</a></h2>
<p>Tester can run without using a database, set property <a href="#tester.nodb">tester.nodb</a> to do so. In that case, this chapter can be skipped.</p>
<p>After each step, Tester can (but does not have to) dump a set of database tables (or views) and compare their contents against expected data, while ignoring (masking out) columns containing sequence numbers, timestamps or other changing data.</p>
<p>The four directories used are <a href="#actual_data_full">actual_data_full</a>, <a href="#actual_data_masked">actual_data_masked</a>, <a href="#expected_data_full">expected_data_full</a> &amp; <a href="#expected_data_masked">expected_data_masked</a> , see their description above. The existence of <a href="#expected_data_masked">expected_data_masked</a> triggers the dumping. Dumping can be temporarily suspended by renaming the directory.</p>
<p>File <a href="#data_definition">data_definition</a>\<a name="tables_masks">tables_masks.csv</a> with its columns <a name="TABLE">TABLE</a>, <a name="ORDER_BY">ORDER_BY </a>, <a name="EXCLUDED_COLUMNS">EXCLUDED_COLUMNS</a> &amp; <a name="WHERE_CLAUSE">WHERE_CLAUSE</a> defines what to compare. Therein column TABLE lists the tables to compare. Column ORDER_BY defines how to sort the dumped data. It&#8217;s a &#8220;,&#8221;-separated list of column names to sort by. Column EXCLUDED_COLUMNS defines which columns to exclude from comparison &#8211; also a &#8220;,&#8221;-separated list of column names. Column WHERE_CLAUSE allows restricting the rows to compare. This is a standard SQL where-clause. To remain portable, no database specific functions should be used. Have a look at chapter <a href="#MaskingOrderingColumnsExample">Masking Columns and ordering records in Billing Example</a>.</p>
<p>File ..\<a href="#defaultsDir">defaults</a>\<a name="tables_defaults.csv">tables_defaults.csv</a> contains the defaults for file <a href="#tables_masks">tables_masks.csv</a>. In file <a href="#tables_masks">tables_masks.csv</a> all except the first column (<a href="#TABLE">TABLE</a>) can contain a &#8220;*&#8221;-character, which causes the corresponding default to be taken from ../<a href="#defaultsDir">defaults</a>/tables_defaults.csv.</p>
<p><a name="EXCLUDE_SORTING_COL">Sorting by filtered columns</a>: Concerning the sorting by columns, which have been filtered out, please take the following into consideration: </p>
<ul>
<li>It is possible, to sort by a column, which is among the EXCLUDED_COLUMNS. But&#8230;</li>
<li><a href="#actual_data_full">actual_data_full</a> is dumped and sorted first.</li>
<li>Then, <a href="#actual_data_masked">actual_data_masked</a> is produced by reading <a href="#actual_data_full">actual_data_full</a> and masking out the undesired columns.</li>
<li>This is efficient, since the database is only read once.</li>
<li>And the positive consequence is, that files in <a href="#actual_data_full">actual_data_full</a> and <a href="#actual_data_masked">actual_data_masked</a> are sorted the same.</li>
<li>But if columns have been excluded, the sorting of <a href="#actual_data_masked">actual_data_masked</a> is not always obvious any more! In such cases, have a look at the corresponding file in <a href="#actual_data_full">actual_data_full</a></li>
<li>These problems can be avoided by not excluding the sorting columns.</li>
</ul>
<p><a name="XP_SORT">XP_SORT</a> &amp; <a name="CollattingSeq">Collating Sequences</a>: If column <a href="#ORDER_BY">ORDER_BY</a> contains the special token &#8220;XP_SORT&#8221;, then Tester will not rely on the database for sorting this table&#8217;s dump but will use the operating system&#8217;s &#8220;sort&#8221; command and sort the lines as a whole. &#8220;sort&#8221; is expected to be on the path. This feature comes handy, if </p>
<ul>
<li>the test sequence and the expected results should be portable between different databases, and</li>
<li>the databases and some of the sorting columns use different collating sequences (language dependent sorting orders), or</li>
<li>the test sequence should not depend on a specific collating sequences at all, or</li>
<li>you do not want to use the sorting features of your database due to performance considerations.</li>
</ul>
<p>Please consider again the above remark concerning <a href="#EXCLUDE_SORTING_COL">Sorting by filtered columns</a>: If columns are filtered out and XP_SORT is used, the ordering of such files in <a href="#actual_data_masked">actual_data_masked</a> might become very hard to understand. The problem is less severe, if at least the first few columns are not masked away.</p>
<p><a name="listed_tables_only.txt">listed_tables_only.txt</a>: listed_tables_only.txt in directory <a href="#data_definition">data_definition</a> is an empty marker file used to tell Tester, that it should process only the tables mentioned in <a href="#tables_masks">tables_masks.csv</a>. In general, a step&#8217;s file <a href="#tables_masks">tables_masks.csv</a> might contain fewer tables than the file ../<a href="#defaultsDir">defaults</a>/<a href="#tables_defaults.csv">tables_defaults.csv</a>. If listed_tables_only.txt is not present, Tester dumps all tables defined in either ../<a href="#defaultsDir">defaults</a>/<a href="#tables_defaults.csv">tables_defaults.csv</a> or in <a href="#data_definition">data_definition</a>/<a href="#tables_masks">tables_masks.csv</a> into directory <a href="#actual_data_full">actual_data_full</a>, but it only dumps the tables listed in file <a href="#tables_masks">tables_masks.csv</a> into directory <a href="#actual_data_masked">actual_data_masked</a> for comparison against <a href="#expected_data_masked">expected_data_masked</a>. If this is not desired and for both (for <a href="#actual_data_full">actual_data_full</a> and for <a href="#actual_data_masked">actual_data_masked</a>) only the tables listed in <a href="#tables_masks">tables_masks.csv</a> should be dumped, this can be forced by creating an (empty) marker-file <a href="#data_definition">data_definition</a>/<a href="#listed_tables_only.txt">listed_tables_only.txt</a>.</p>
<p>Tester can also load data into a database. It does so at the second step or &#8211; if <a href="#Prop.from">tester.from</a> is defined &#8211; at the step just before <a href="#Prop.from">tester.from</a>. You might also use <a href="#Uploader">Uploader</a> to load data. <a href="#Uploader">Uploader</a> is simply a test sequence with a single loading step. It calls Tester.</p>
<p>If property <a href="#Prop.noload">tester.noload</a> is defined, Tester does not recreate the database nor load data. </p>
<h2><a name="ChapVolumePerformance">Data Volume and Performance</a></h2>
<p>To load data into a database, Tester is limited by main memory. It is not planned to change this.</p>
<p>The amount of main memory can be configured in <a href="#Tester.bat">Tester.bat</a>.</p>
<p>But to dump data out of a database, Tester is NOT limited by main memory. This means, tester can work on a low memory foot print.</p>
<p>The performance to dump data out of an Oracle database is comparable to what for example SQL-Plus accomplishes. The dumping of data is actually faster than SQL-Plus. Tester dumps it in about 2/3 of the time SQL-Plus takes. But recall, Tester masks out unused columns and then compares data, which in total takes more time.</p>
<h2><a name="ChapExecuteATest">Execution of a Test</a></h2>
<h3><a name="ChapCallStack">The Call Stack</a></h3>
<p>Executing a test involves three applications (the application under test, the test sequence and Tester) and their properly coordinated configuration. To have a high flexibility, there is an entire script framework and a few <a href="#configurationfiles">Configuration Files</a>. Each test sequence has a execution script &lt;Testsequence&gt;.bat, Tester itself has a <a href="#Tester.bat">Tester.bat</a> script and each test step has a <a href="#Step.bat">Step.bat</a>, which allows calling arbitrary programs under test. To explain this, the following overview will be discussed.</p>
<p><img style="width: 572px; height: 531px;" alt="Call Stack" title="Call Stack" src="wp-content/gallery/varia/Callstack.jpg"></p>
<p>A test sequence is an ordinary program, which has a configuration and is working on some data. The test sequence program is a .bat Script under Windows, or a script without extension under Linux. Script, data and configuration are contained in a single directory. This allows for easy packing, moving or versioning of test sequences. Above, the hexagon symbol is used to denote directories. Directories &#8216;contain&#8217; other directories, scripts and data files.</p>
<p>If a test sequence is newly created or moved, its configuration must be adapted. The test sequence must know where to find the installation directory of Tester (TESTER_HOME) and where to find the application under test and possibly the database account to use. <a href="#TESTER_HOME">TESTER_HOME</a> is defined in the test sequence script. For other configuration settings there are multiple possibilities which should be chosen carefully:</p>
<ul>
<li>Environment variables defined outside &lt;Testsequence&gt;.bat are recommended, if they are valid for the person executing the test.</li>
<li>Environment variables defined within &lt;Testsequence&gt;.bat are recommended, if they are valid for an entire test sequence, but not for other sequences.</li>
<li><a href="#Tester.bat">Tester.bat</a> also contains a few settings, which might be handy to use. They are typically recommended for settings describing the local installations on the machine on which the tests are executed. Typically it contains paths to installed software like some database client software (Oracle Net or sqlplus for example) if this information is useful for all test sequences.</li>
<li>Like a typical java application, Tester reads .properties-files. Tester gets the jdbc connect string and the database account from .properties-files. Which files to read can be passed as Arguments (<a href="#CONFIGDIR">CONFIGDIR</a> and <a href="#PROPFILES">PROPFILES</a>) to Tester.bat. Some sensible defaults should be defined within Tester.bat directly. Tester has a useful <a href="#ChapIndirection">indirection Feature</a>, which allows sharing the database access configuration with the application under test, see next subchapter.</li>
<li><a href="#Step.bat">Step.bat</a> might override most configurations with step specific configurations before it calls the application under test. But it should not override jdbc connect string and database account, since this would circumvent Tester.</li>
</ul>
<p>The rhombus symbols of the above figure are used to denote configuration variables. In the text, we use the notation &lt;a&gt;. The most important settings are: </p>
<ul>
<li>&lt;a&gt;:Tester gets its configuration in the form of java properties on the command line or from .properties files. Its indirection feature allows to share the database configuration with the application under test, see <a href="#ChapProperties">Properties / Parameters</a>.</li>
<li>&lt;b&gt;:<a href="#TESTER_HOME">TESTER_HOME</a>, the installation directory of Tester. Each &lt;Testsequence&gt;.bat script must define this variable, to find Tester.</li>
<li>&lt;c&gt;: Each test sequence is packed into its own directory. There is no dependency on this. The directory can be moved freely.</li>
<li>&lt;d&gt;: The test steps are directories within the test sequence directory.</li>
<li>&lt;e&gt;: The jdbc driver is typically defined in a <a href="#myStandard.properties">myStandard_*.properties</a> file. Tester searches the -D arguments passed to the java virtual machine and then the files defined by <a href="#Prop.propertyfiles">tester.propertyfiles</a> on the java classpath. See also chapter <a href="#ChapIndirection">indirection Feature</a>.</li>
<li>&lt;f&gt;: Username and password are typically found in a <a href="#mySecured.properties">mySecured_*.properties</a> file. The lookup mechanism is the same as for the jdbc-driver.</li>
<li>&lt;g&gt;: The path to the application under test can be hard coded within the <a href="#Step.bat">Step.bat</a> scripts or &#8211; recommended &#8211; you define your own variables which you then reference in the Step.bat. Your variables might be defined in the user environment, in &lt;Testsequence&gt;.bat or in Tester.bat, arguments see above.</li>
</ul>
<p>Now, the sequence of application calls is discussed. In the figure above the numbers in circles roughly denote the order of calls. In the text, we use (). The sequence of application calls is as follows: </p>
<ul>
<li>&lt;Testsequence&gt;.bat is started.</li>
<li>(1) &lt;Testsequence&gt;.bat calls <a href="#Tester.bat">Tester.bat</a> or it might call Tester via the java virtual machine (see next subchapter).</li>
<li>(2) Tester might load data onto the database (unless <a href="#Prop.noload">tester.noload</a> is defined).</li>
<li>(3) Then Tester calls the <a href="#Step.bat">Step.bat</a> scripts of the test steps of your sequence, one after the other, possibly restricted by <a href="#Prop.from">tester.from</a> &amp; <a href="#Prop.to">tester.to</a>. Step.bat can stop the execution of the sequence by returning exit-code 99 or 98, see <a href="#Step.bat">Step.bat</a></li>
<li>(4) Step.bat calls the application under Test. It provides input to the application.</li>
<li>(5) The application might access and modify the database and produce output. Step.bat should copy the latter back to the teststep directory for the comparison (or at least dump it to stdout).</li>
<li>(6) After termination of the application and of a Step.bat, Tester might download database data. Afterwards, it compares database data and the output of the application against expected data.</li>
</ul>
<h3><a name="ChapCallingTester">Calling Tester</a></h3>
<p>There are two ways to call Tester:</p>
<p><a name="Tester.bat">Tester.bat </a>: Tester can be called by use of Tester.bat, which takes a couple of arguments. This is the recommended and simpler way. If it is called without arguments, the list of available parameters is printed. Tester is the corresponding Linux bash-Script. The following documentation only refers to Tester.bat and implicitly means the bash-Script in the case of a Linux environment. <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> is a documented example of how to call Tester.bat.</p>
<p>Tester might also be called directly by use of the java virtual machine. The call looks like this:</p>
<p><code>java -cp %CP% %ARGS% com.stefanbuehlmann.tester.Tester</code></p>
<p><a name="classpath">classpath </a>: The classpath %CP% must contain: Tester.jar, dbunit-2.1.jar, junit.jar, commons-io-1.0.jar, a jar containing your jdbc-driver and a path to your <a href="#configurationfiles">Configuration Files</a>, see <a href="#Requirements">Requirements</a> and <a href="#Installation">Installation</a>.</p>
<p><a name="configurationfiles">Configuration Files</a>: Arguments %ARGS% can either be given as java properties on the command line (with leading -D) or in standard java .properties-files. The latter is the reason, why %CP% must contain a path to your configuration files. The following subchapter lists all available properties.</p>
<p><a name="environmentvariables">Environment Variables</a> defined in the environment at the time Tester is called, are passed on to <a href="#Step.bat">Step.bat</a>.</p>
<p>At the end of a run of Tester, either the message &#8220;###&gt;normal end&#8221; is displayed, indicating, that no errors or differences were found, or a summary of steps is given, with the first difference per step.</p>
<h3><a name="ChapProperties">Properties / Parameters</a>/<a name="ChapIndirection">indirection Feature</a></h3>
<p>Tester has a useful properties-indirection feature for some of its properties. If a property of the form tester.propertyname.&lt;x&gt; exists, Tester takes its value to search for another property. The value of this second property is then taken instead of the value of the corresponding property tester.&lt;x&gt;. Example, if you define -Dtester.propertyname.jdbc.driver=my.jdbc.driver on the command line, Tester infers the jdbc-driver by looking at property my.jdbc.driver. If tester.propertyname.jdbc.driver is not present, or if my.jdbc.driver is missing, the normal property tester.jdbc.driver is taken.</p>
<p>With this properties-indirection feature, Tester can be configured to use the same .properties files as the application under test (if this is also a java application). This makes sense especially for the database configuration properties.</p>
<p><a name="configs">configs</a>, <a name="myRedirectedStandard.properties">myRedirectedStandard.properties</a>, <a name="myStandard.properties">myStandard_*.properties</a> &amp; <a name="mySecured.properties">mySecured_*.properties</a>: <a href="#ChapBillingExample">Billing Example</a> comes with directory configs, which contains the <a href="#configurationfiles">Configuration Files</a> myRedirectedStandard.properties and DB-specific myStandard_*.properties and mySecured_*.properties. myRedirectedStandard.properties defies the property names to use and myStandard_*.properties defines general jdbc-Driver settings, where about mySecured.properties defines username and password to use. mySecured.properties is meant to be kept at a secure place and can be different for different developers.</p>
<p>The following properties are available: </p>
<table>
<tbody>
<tr>
<td><a name="tester.nodb">tester.nodb</a></td>
<td>If tester.nodb is set, Tester ignores all database-specific properties and assumes, Tester is run without connection to a database. That means, no upload or download and compare of database data is done. tester.nodb is optional and can not be indirected.</td>
</tr>
<tr>
<td><a name="Prop.propertyfiles">tester.propertyfiles </a></td>
<td>a &#8220;,&#8221;-separated list of configuration file names (Java .properties-files) which must be accessible via the <a href="#classpath">classpath</a>. &#8220;.properties&#8221; is appended to the filenames. tester.propertyfiles is mandatory and can not be indirected. You might have a look at how <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> defines the variable <a href="#PROPFILES">PROPFILES</a> and how it passes this to <a href="#Tester.bat">Tester.bat</a>, which then assigns it to tester.propertyfiles.</td>
</tr>
<tr>
<td><a name="Prop.from">tester.from </a></td>
<td>Test step to start with, see <a href="#ChapSeqOfSteps">Sequences of Test Steps</a>. tester.from is optional and can not be indirected. Again, as an example you might look at <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> , which defines variable <a href="#TEST_FROM">TEST_FROM</a> and passes it on to <a href="#Tester.bat">Tester.bat</a>.</td>
</tr>
<tr>
<td><a name="Prop.to">tester.to </a></td>
<td>Test step to end at, see <a href="#ChapSeqOfSteps">Sequences of Test Steps</a>. tester.to is optional and can not be indirected. Again you might look at <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a>, which uses variable <a href="#TEST_TO">TEST_TO</a> for this purpose.</td>
</tr>
<tr>
<td><a name="Prop.noload">tester.noload</a></td>
<td>If defined, the <a href="#FirstStep">First Step</a> (database recreation) and database loading (either the <a href="#SecondStep">Second Step</a> or the step immediately before <a href="#Prop.from">tester.from</a> ) will not occur. This is usually helpful, if a previous test step just ran successfully (without any differences) and you want to continue with the next test steps without resetting the database again. If there were differences at the last step, using <a href="#Prop.noload">tester.noload</a><br />
is not recommended. See also <a href="#ChapSeqOfSteps">Sequences of Test Steps</a>. <a href="#Prop.noload">tester.noload</a> is optional and can not be indirected. Again you might look at <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> and variable <a href="#TEST_NOLOAD">TEST_NOLOAD</a>.</td>
</tr>
<tr>
<td><a name="Prop.jdbc.driver">tester.jdbc.driver </a></td>
<td>The jdbc-driver to use. Example: tester.jdbc.driver=com.mysql.jdbc.Driver. tester.jdbc.driver can be indirected. For an example, have a look at the configuration files in directory <a href="#configs">configs</a>.</td>
</tr>
<tr>
<td><a name="Prop.jdbc.connection"> tester.jdbc.connection </a></td>
<td>The connection URL to your database, as defined by your jdbc-driver. For example, for MYSQL this would be something like: tester.jdbc.connection=jdbc:mysql://[hostname][,failoverhost...][:port]/[dbname][&amp;param1=value1][&amp;param2=value2]&#8230; tester.jdbc.connection can be indirected.</td>
</tr>
<tr>
<td><a name="Prop.jdbc.database">tester.jdbc.database</a></td>
<td>tester.jdbc.database is not used by Tester, but (to allow consistent handling of all possible parameters) it is passed on to your <a href="#Step.bat">Step.bat</a>. tester.jdbc.database can be indirected.</td>
</tr>
<tr>
<td><a name="Prop.jdbc.schema">tester.jdbc.schema</a></td>
<td>The database schema to be used. tester.jdbc.schema can be indirected.</td>
</tr>
<tr>
<td><a name="Prop.jdbc.user">tester.jdbc.user</a></td>
<td>You guess it. tester.jdbc.user can be indirected.</td>
</tr>
<tr>
<td><a name="Prop.jdbc.password">tester.jdbc.password </a></td>
<td>You guess it. tester.jdbc.password can be indirected.</td>
</tr>
<tr>
<td><a name="Prop.others"> others&#8230; </a></td>
<td>In addition you might pass on any properties to dbunit, junit, commons-io, or your jdbc-driver.</td>
</tr>
</tbody>
</table>
<h2><a name="ChapBillingExample">Billing Example</a></h2>
<p>Billing Example demonstrates the features of Tester with some &#8220;real&#8221; data, see chapter <a href="#ChapBillingExampleFeatures">Features demonstrated by Billing Example</a>. It allows you to play around with a simple situation, with a small sequence of test steps.</p>
<p>Billing Example is tested for MySQL, Oracle and DB2. (it shouldn&#8217;t be too complicated to adapt to other databases).</p>
<h3><a name="ChapBillingExampleDM">The Data Model of Billing Example</a></h3>
<p><img style="width: 362px; height: 119px;" alt="Data Model of Billing Example" title="Data Model of Billing Example" src="wp-content/gallery/varia/Billing_Example.jpg"></p>
<p>This data model should be self-describing. Have a look at the detailed table definitions in Billing_Example\0000\create_db_*.sql.</p>
<p>In step 0001 some data is loaded to start the tests with. </p>
<p>Step 0002 shows, how the addition of two CUSTOMER records can be tested.</p>
<p>Step 0003 shows additions of ADDRESSes and BILLs with POSITIONs and how to test such insertions. It also shows an update of an ADDRESS record.</p>
<h3><a name="ChapBillingExampleRunning">Running Billing Example</a></h3>
<p><a name="Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> &amp; <a name="InstallBillingExample"> Installing Billing Example</a>: Billing_Example_Tester.bat is the script to run the Billing Example. Billing_Example_Tester is the corresponding Linux bash-Script. The following documentation only refers to Billing_Example_Tester.bat and implicitly means the bash-Script in the case of a Linux environment.</p>
<p>Billing_Example_Tester.bat has the following additional variables, which you might change before running it:
<ul style="margin-top: 0cm;" type="disc">
<li><a name="DB_TYPE">DB_TYPE </a>: Edit DB_TYPE to MYSQL, DB2 or ORACLE, depending on what your database is. This allows switching later. DB_TYPE is used to choose among the jdbc-drivers or to execute database specific scripts. For example <a href="#ChapBillingExample">Billing Example</a> step 0000 uses it to switch between different db-creation scripts.</li>
<li><a name="TEST_FROM">TEST_FROM</a>: Set variable TEST_FROM to the first step you want to run. Leave it empty, if you want to start from the beginning. (TEST_FROM corresponds to the property <a href="#Prop.from">tester.from</a>)</li>
<li><a name="TEST_TO">TEST_TO</a>: Set variable TEST_TO to the last step you want to run. Leave it empty, if you want to run to the end. (TEST_TO corresponds to the property <a href="#Prop.to">tester.to</a>)</li>
<li><a name="TEST_NOLOAD">TEST_NOLOAD</a>: Set variable TEST_NOLOAD to the empty value, if you want Tester to reset the database, set it to any non-empty value otherwise. TEST_NOLOAD corresponds to <a href="#Prop.noload">tester.noload</a>), where you find more information if needed.</li>
</ul>
<p>You might now run Billing Example by double-clicking on <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a>.</p>
<p>If <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> is unchanged since <a href="#Installation">Installation</a>, this will run all test steps of Billing Example. But it is possible to run only a subset of test steps. The following subchapters show some examples:</p>
<ul style="margin-top: 0cm;" type="disc">
<li><a href="#ChapBillingRunA">(a) Running Billing Example with all Steps</a></li>
<li><a href="#ChapBillingRunB">(b) Running Billing Example Steps 0000 to 0002</a></li>
<li><a href="#ChapBillingRunC">(c) Running Billing Example starting at Step 0003 without resetting the database</a></li>
<li><a href="#ChapBillingRunD">(d) Running Billing Example Step 0003 only</a></li>
</ul>
<h4><a name="ChapBillingRunA">(a) Running Billing Example with all Steps</a></h4>
<p><strong>Situation:</strong> You want to re-run all your test steps (or it is unclear, whether <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> was changed since the installation of Tester).</p>
<p>To run all steps, you have to edit <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> and to set all three variables to the empty string as shown here (this is how a newly installed <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> looks like):</p>
<div><code>set <a href="#TEST_FROM">TEST_FROM</a>=<br />
set <a href="#TEST_TO">TEST_TO</a>=<br />
set <a href="#TEST_NOLOAD">TEST_NOLOAD</a>=<br />
</code></div>
<p>Having done so, you simply double-click <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a>. This should run the example and finish with message &#8220;###&gt;normal end&#8221;. </p>
<h4><a name="ChapBillingRunB">(b) Running Billing Example Steps 0000 to 0002</a></h4>
<p><strong>Situation:</strong> Assume, you changed some code in the program called by <a href="#Step.bat">Step.bat</a> of step 0002 and you are sure, this change does not affect other steps. It would be a waste of time to run all steps again.</p>
<p>To run only steps 0000 to 0002, edit the three variables in <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> as follows: </p>
<div><code>set <a href="#TEST_FROM">TEST_FROM</a>=<br />
set <a href="#TEST_TO">TEST_TO</a>=0002<br />
set <a href="#TEST_NOLOAD">TEST_NOLOAD</a>=<br />
</code></div>
<p>Then double-click <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> and verify, that 0000 up to 0002 did run, but 0003 did not run.</p>
<h4><a name="ChapBillingRunC"> (c) Running Billing Example starting at Step 0003 without resetting the database</a></h4>
<p><strong>Situation:</strong> Assume, you just ran steps 0000 up to 0002 without error ? for example you just did (b) &#8211; and you want to run step 0003 now, without running all the way from 0000 to 0003 and without refilling the database.</p>
<p>To do so, edit the three variables in <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> as follows:</p>
<div><code>set <a href="#TEST_FROM">TEST_FROM</a>=0003<br />
set <a href="#TEST_TO">TEST_TO</a>=<br />
set <a href="#TEST_NOLOAD">TEST_NOLOAD</a>=anything<br />
</code></div>
<p>Then double-click <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> and verify, that 0000 up to 0002 did not run, no loading occurred, and only 0003 did run without errors.</p>
<h4><a name="ChapBillingRunD">(d) Running Billing Example Step 0003 only</a></h4>
<p><strong>Situation:</strong> Assume, you want to run step 0003 only, but your database is not in the state as at the end of step 0002 &#8211; for example you just did (c) which left the database in a state as of the end of step 0003 &#8211; or the state of the database is unknown. So Tester should set the database to the state as expected after step 0002.</p>
<p>To do so, edit the three variables in <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> as follows:</p>
<div><code>set <a href="#TEST_FROM">TEST_FROM</a>=0003<br />
set <a href="#TEST_TO">TEST_TO</a>=0003<br />
set <a href="#TEST_NOLOAD">TEST_NOLOAD</a>=<br />
</code></div>
<p>Then double-click <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> and verify, that step 0000 did run, the data of step 0002 was loaded and finally only 0003 did run without errors. Step 0001 should not have run.</p>
<h3><a name="ChapBillingExampleFeatures">Features demonstrated by Billing Example</a></h3>
<p>Billing Example demonstrates quite a few features of Tester. The following aspects might be worth looking at (paths are given relative to .\Billing_Example\):</p>
<ul style="margin-top: 0cm;" type="disc">
<li>0000\<a href="#Step.bat">Step.bat</a> is a typical Step.bat. You might use it as a template. You might want to read the comments (lines starting with &#8220;REM&#8221;). 0000\<a href="#Step.bat">Step.bat</a> calls SQL-script 0000\create_db_MYSQL.sql, which re-creates an empty database corresponding to the data model of the example by dropping and recreating the tables.</li>
<li>0001\<a href="#expected_data_full">expected_data_full</a> contains the data to load initially.</li>
<li>0002\<a href="#Step.bat">Step.bat</a> is a typical <a href="#Step.bat">Step.bat</a>. Normally, a <a href="#Step.bat">Step.bat</a> would call your application under test and at the end of the step, Tester would compare output and database content. But since the example does not include a real application to test, 0002\<br />
<a href="#Step.bat">Step.bat</a> simulates an application by modifying the database itself by applying modify_db_0002_MYSQL/DB2/ORACLE.sql.</li>
<li><a name="RegularExpressionsExample"> Regular Expressions in Billing Example</a>: Having run Billing Example at least once, you might want to compare the various log files in 0002. Compare 0002\<a href="#actual_full.log">actual_full.log</a> with 0002\ <a href="#actual_masked.log">actual_masked.log</a> or 0002\ <a href="#expected_full.log">expected_full.log</a> with 0002\<a href="#expected_masked.log">expected_masked.log</a> to see the effects of the global replacements defined by the <a href="#RegularExpressions">Regular Expressions</a> in ..\<a href="#defaultsDir">defaults</a>\<a href="#log_defaults.csv">log_defaults.csv</a> and 0002\<a href="#data_definition">data_definition</a>\<a href="#log_masks.csv">log_masks.csv</a>. Try playing around with ..\<a href="#defaultsDir">defaults</a>\<a href="#log_defaults.csv">log_defaults.csv</a> and 0002\<a href="#data_definition">data_definition</a>\<a href="#log_masks.csv">log_masks.csv</a> to see how it works.</li>
<li>Compare 0002\<a href="#expected_full.log">expected_full.log</a> to 0002\<a href="#actual_full.log">actual_full.log</a>, to see how the global replacements can mask away database and connection specific stuff.</li>
<li><a name="MaskingOrderingColumnsExample">Masking Columns and ordering records in Billing Example</a>: Compare the files in 0002\<a href="#expected_data_full">expected_data_full</a> and 0002\<a href="#expected_data_masked">expected_data_masked</a>, to learn about excluding columns, ordering or restricting data when comparing database content. This is defined in ..\<a href="#defaultsDir">defaults</a>\<a href="#tables_defaults.csv">tables_defaults.csv</a> and 0002\<a href="#data_definition">data_definition</a>\<a href="#tables_masks">tables_masks.csv</a>. You might play around with these files. You might verify your doing by looking at the &#8220;###&gt; query: select * from&#8230;&#8221;-lines written to stdout.</li>
<li>In ..\<a href="#defaultsDir">defaults</a>\<a href="#tables_defaults.csv">tables_defaults.csv</a> and its column <a href="#EXCLUDED_COLUMNS">EXCLUDED_COLUMNS</a> you can see, that the primary keys ID are not dumped, because we assume, that they are generated by sequences. But remark, the example is not consequent, it still dumps these values when they are used as foreign keys, example POSITION.ARTICLE. Try to correct this.</li>
<li>..\<a href="#defaultsDir">defaults</a>\<a href="#tables_defaults.csv">tables_defaults.csv</a> and its column <a href="#ORDER_BY">ORDER_BY</a> is an example of how to order data. Files must always be in the same order to be compared.</li>
<li>See what happens, if you add a <a href="#WHERE_CLAUSE">WHERE_CLAUSE</a> in that file.</li>
<li>ADDRESS.STREET_2 is a null-able column. Have a look at the corresponding .csv-files if you insert or delete a value.</li>
<li>Have a look at other data types, e.g. BILL.ORDER_DATE, ARTICLE.PRICE, ARTICLE.PROMOTIONCODE&#8230;</li>
<li>Force a sql-Error by attempting to insert a duplicate record in script 0003\modify_db_0003_MYSQL.sql. Tester will say on stdout: &#8220;###&gt;ERROR: files &#8216;.\0003\<a href="#actual_masked.log">actual_masked.log</a>&#8216; and &#8216;.\0003\<a href="#expected_masked.log">expected_masked.log</a>&#8216; are not equal.&#8221; Compare these two files. Verify, the script 0003\modify_db_0003_MYSQL.sql did not run to completion, there are &#8220;ERR&gt;&#8221;-lines in the log file and <a href="#Step.bat">Step.bat</a> exited with error code 1.</li>
<li>Since MySQL does not yet support views with version 4.1 the example can not show how to compare view contents (in Oracle this would be possible). 0003\modify_db_0003_MYSQL.sql uses a workaround to verify the result of a complex sql-statement. It dumps the result of the select to stdout.</li>
<li><a name="Linetypes">Line types</a>: Have a look at the various line types of the log and the console. There are:
<ul style="margin-top: 0cm;" type="disc">
<li>&#8220;###&gt;######&#8230;.######&#8221;-lines: These are lines written by Tester to separate the various test steps.</li>
<li>&#8220;###&gt;&#8221;-lines: These are lines written by Tester itself.</li>
<li>&#8220;OUT&gt;&#8221;-lines: These lines are stdout lines of your <a href="#Step.bat">Step.bat</a> or your application under test.</li>
<li>&#8220;ERR&gt;&#8221;-lines: These lines are stderr lines of your <a href="#Step.bat">Step.bat</a> or your application under test.</li>
<li>&#8220;INP&gt;&#8221;-lines: These lines are stdin lines of your <a href="#Step.bat">Step.bat</a> or your application under test.</li>
</ul>
<p>For example, it would be easy to search the logs for any inputs made.</li>
<li>Tester stops immediately if a fatal error occurs or if the test steps are not set up correctly, but it continues when differences in log files or database content are found. This allows handling multiple differing steps in one run. Try to force both types of errors. Remark, when comparing database contents Tester will stop after the first difference, but you can use your diff-tool to find all.</li>
<li><a name="Billing_Example_Clean.bat">Clean.bat</a>: Since calling Tester always produces a lot of new files, Billing_Example_Clean.bat is provided. This simple script deletes all unneeded files. Just double-click Billing_Example_Clean.bat to run it. Since different runs of Tester might create different sets of files, Billing_Example_Clean.bat might not find all and yield errors. You simply ignore these. See also <a href="#FAQ_VersionControl">Which files should go under version control?</a></li>
</ul>
<h2><a name="ChapFAQs">FAQs</a></h2>
<h3><a name="FAQ_Downloader">How do I dump the contents of a database?</a></h3>
<p><a name="Downloader">Downloader</a>: Use the example Downloader (consisting of directory Downloader, its subdirectories and the scripts Downloader.bat and Downloader_Clean.bat &#8211; and Downloader and Downloader_Clean under Linux).</p>
<p>Define the list of tables you want to download in Downloader\<a href="#defaultsDir">defaults</a>\<a href="#tables_defaults.csv">tables_defaults.csv</a>. Define <a href="#ORDER_BY">ORDER_BY</a>, <a href="#EXCLUDED_COLUMNS">EXCLUDED_COLUMNS</a> &amp; <a href="#WHERE_CLAUSE">WHERE_CLAUSE</a> if required, or leave them empty. You can use file ..\<a href="#defaultsDir">defaults</a>\<a href="#tables_defaults.csv">tables_defaults.csv</a> of an existing test sequence if you want, or create it from scratch.</p>
<p>Each table must also be listed in Downloader\0002\<a href="#data_definition">data_definition</a>\<a href="#tables_masks">tables_masks.csv</a>. Also this file can be copied from an existing test sequence, if available.</p>
<p>Downloader comes with two table lists corresponding to the <a href="#ChapBillingExample">Billing Example</a>, to have you up and running quicker.</p>
<p>Now you are ready to run Downloader.bat. You find the downloaded data in Downloader\0002\<a href="#actual_data_full">actual_data_full</a> &amp; Downloader\0002\<a href="#actual_data_masked">actual_data_masked</a>.</p>
<p>Downloader.bat should only report one error, saying, that there are differences between &#8216;.\0002\<a href="#expected_data_masked">expected_data_masked</a>&#8216; &amp; &#8216;.\0002\<a href="#actual_data_masked">actual_data_masked</a>&#8216;. This is expected, since the creator of Downloader did not know anything about the data to be downloaded.</p>
<p>Remark how simple Downloader is. The directories Downloader\0000 and Downloader\0001 are empty. They are required to make Tester think there are steps before 0002. But they are not used, because Downloader.bat defines <a href="#TEST_NOLOAD">TEST_NOLOAD</a>. They can remain empty. And there is no Downloader\0002\<a href="#Step.bat">Step.bat</a>, Downloader\0002 only contains <a href="#data_definition">data_definition</a>\<a href="#tables_masks">tables_masks.csv</a> and there are no .log-files produced, since there is no <a href="#Step.bat">Step.bat</a>.</p>
<p>As you can see, downloading a database is very simple. Just list the tables you want in ..\<a href="#defaultsDir">defaults</a>\<a href="#tables_defaults.csv">tables_defaults.csv</a> &amp; Downloader\0002\<a href="#data_definition">data_definition</a>\<a href="#tables_masks">tables_masks.csv</a> and run Downloader.bat. It does not make sense to create a special Tester option for downloading. </p>
<h3><a name="FAQ_Uploader">How do I upload data into a database with Tester?</a></h3>
<p><a name="Uploader">Uploader</a>: Use example Uploader (consisting of directory Uploader, its subdirectories and the scripts Uploader.bat and Uploader_Clean.bat &#8211; and Uploader and Uploader_Clean for Linux).</p>
<p>You must first re-create an empty database. This can be achieved with running step 0000 (only 0000) of your step sequence. To try it, you might run step 0000 of <a href="#ChapBillingExample">Billing Example</a>.</p>
<p>Place any .csv-files you want to load into Uploader\0001\<a href="#expected_data_full">expected_data_full</a>. You don&#8217;t have to define a list of tables to be uploaded, because you implicitly do so by putting the .csv-files into Uploader\0001\<a href="#expected_data_full">expected_data_full</a>. For an example to try, there is file ADDRESSTYPE.csv included, corresponding to the Billing Example. It is here to demonstrate how uploading works. But most probably you will delete it as soon as you understood how it works, since this record might not make sense in your data model (it only goes with the example).</p>
<p>Now run Uploader.bat and watch the database. Simple, isn&#8217;t it? </p>
<h3><a name="FAQ_StartAnewSeq">How do I best start creating a new sequence of test steps?</a></h3>
<ul>
<li>Start with a copy of <a href="#ChapBillingExample">Billing Example</a>, but delete its step 0003.</li>
<li>Rename the directory of <a href="#ChapBillingExample">Billing Example</a>, <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> &amp; <a href="#Billing_Example_Clean.bat">Clean.bat</a> to your liking.</li>
<li>Edit your copy of <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a> corresponding to the comments.</li>
<li>Then make sure, your <a href="#FirstStep">First Step</a> creates the database tables you want. (Set <a href="#TEST_TO">TEST_TO</a> to the <a href="#FirstStep">First Step</a> to do so). Use <a href="#Step.bat">Step.bat</a> of the <a href="#ChapBillingExample">Billing Example</a> as a template.</li>
<li>List all your database tables in ../<a href="#defaultsDir">defaults</a>/<a href="#tables_defaults.csv">tables_defaults.csv</a> &amp; <a href="#data_definition">data_definition</a>\<a href="#tables_masks">tables_masks.csv</a>. Try to define correct values for <a href="#ORDER_BY">ORDER_BY</a> &amp; <a href="#EXCLUDED_COLUMNS">EXCLUDED_COLUMNS</a> in <a href="#tables_defaults.csv">tables_defaults.csv</a> from the beginning, this helps to avoid a lot of subsequent work. <a href="#WHERE_CLAUSE">WHERE_CLAUSE</a> might be set up later. In initial versions of the <a href="#tables_masks">tables_masks.csv</a> files you should use the defaults of <a href="#tables_defaults.csv">tables_defaults.csv</a>, by using &#8220;*&#8221; entries for the columns <a href="#ORDER_BY">ORDER_BY</a>, <a href="#EXCLUDED_COLUMNS">EXCLUDED_COLUMNS</a> &amp; <a href="#WHERE_CLAUSE">WHERE_CLAUSE</a>.</li>
<li>Now the question is, how you get the initial data into .csv-files to be loaded by the second step. One solution is, to manually enter data to the database by use of another tool and then us <a href="#Downloader">Downloader</a> to capture that data. Another possibility is to use <a href="#Downloader">Downloader</a> on the empty database to create skeletons of .csv-files (containing a header row with attribute names only), which you then edit manually. In both cases, you then place the produced .csv-files to <a href="#SecondStep">Second Step</a>\<a href="#expected_data_full">expected_data_full</a>.</li>
<li>If your first two steps re-create the database and fill it with initial data, you are ready to do the first real test step. Again, you might use <a href="#Step.bat">Step.bat</a> of the <a href="#ChapBillingExample">Billing Example</a> as a template. To create the expected results, refer to chapter <a href="#FAQ_DefineExpected">How do I best create the expected results?</a>.</li>
<li>It needs a little bit of experience to use Tester. You have to avoid, that each simple change you do, ripples through all your steps. Wise planning and collecting multiple changes together helps against this phenomenon. Also the last alternative of chapter <a href="#FAQ_DefineExpected">How do I best create the expected results?</a> usually helps a lot. So try to plan what you intend to change.</li>
</ul>
<h3><a name="FAQ_DefineExpected">How do I best create the expected results?</a></h3>
<p>How do I best create the <a href="#expected_full.log">expected_full.log</a> and the files in <a href="#expected_data_full">expected_data_full</a> and <a href="#expected_data_masked">expected_data_masked</a>?</p>
<p>There are the following ways to define expected results:
<ul style="margin-top: 0cm;" type="disc">
<li>The best thing is to define expected results by hand, because this way you really think about how the results should look like. Unfortunately, often this is too cumbersome.</li>
<li>Often, you can get expected results from an older system.</li>
<li>It&#8217;s not optimal, but acceptable, if you assume actual results are correct and whenever you find an error, you correct the expected results. That means, you copy the <a href="#actual_full.log">actual_full.log</a> over <a href="#expected_full.log">expected_full.log</a> and you copy the files of <a href="#actual_data_full">actual_data_full</a> over <a href="#expected_data_full">expected_data_full</a> and you copy the files of <a href="#actual_data_masked">actual_data_masked</a> over <a href="#expected_data_masked">expected_data_masked</a>. But you have to be aware, that this approach is dangerous. So be very careful, that no errors are introduced into your expected data. Use a good diff-Tool (see <a href="#Requirements">Requirements</a>). If you choose this approach, expected data should become better all the time (because you only remove errors).</li>
<li>You can set up your test steps and only compare the log files, which you manually verify. When all test steps run without error, you might use the above approach for the .csv-files. Having produced the .csv-files, you might want to review them in detail.</li>
</ul>
<h3><a name="FAQ_VersionControl">Which files should go under version control?</a></h3>
<p><a name="VersionControl">Version Control</a>: The following files and directories should be under version control:</p>
<ul style="margin-top: 0cm;" type="disc">
<li><a href="#Step.bat">Step.bat</a> and files used by <a href="#Step.bat">Step.bat</a></li>
<li><a href="#expected_full.log">expected_full.log</a></li>
<li>directories <a href="#expected_data_masked">expected_data_masked</a> &amp; <a href="#expected_data_full">expected_data_full</a> including their .csv-files</li>
<li>directories <a href="#actual_data_masked">actual_data_masked</a> &amp; <a href="#actual_data_full">actual_data_full</a> but without their contents</li>
<li>directories <a href="#data_definition">data_definition</a> &amp; ..\<a href="#defaultsDir">defaults</a>including contents</li>
</ul>
<p>The following files and directories should NOT be under version control (have a look at <a href="#Billing_Example_Clean.bat">Clean.bat</a>, which deletes all these):
<ul style="margin-top: 0cm;" type="disc">
<li><a href="#actual_full.log">actual_full.log</a></li>
<li><a href="#actual_masked.log">actual_masked.log</a></li>
<li><a href="#expected_masked.log">expected_masked.log</a></li>
<li>.csv-files of directories <a href="#actual_data_masked">actual_data_masked</a> &amp; <a href="#actual_data_full">actual_data_full</a></li>
</ul>
<h3><a name="FAQ_SecondStep">Why does the second step only contain expected_data_full but no Step.bat?</a></h3>
<p>The <a href="#FirstStep">First Step</a> re-creates an empty database. The step 0001 &#8211; which is the <a href="#SecondStep">Second Step</a> &#8211; is used to initially load data into the database.</p>
<p>The first and second step can not be combined, because the first step is used alone to re-create the database regardless whether test execution starts at the beginning or at later steps.</p>
<p>Loading data into an empty database should work the same regardless whether starting at the beginning or starting at some later steps. So to load the database, the system takes the <a href="#expected_data_full">expected_data_full</a> directory of the step immediately before the starting step (<a href="#Prop.from">tester.from</a>). This is 0001, in case all steps are run, and it is step n-1 in case the tests start with step n.</p>
<p>If step 0001 would contain and execute a <a href="#Step.bat">Step.bat</a>, data loading would work different in case all steps are run, and in case the tests start with a later step.</p>
<p>These are the reasons for these asymmetries.</p>
<h2><a name="Download">Download Tester</a></h2>
<table>
<tbody>
<tr>
<td>build 593 <a class="downloadlink" href="http://www.stefanbuehlmann.com/s/wp-content/plugins/download-monitor/download.php?id=4" title="Version593 downloaded 41 times" >Tester (41)</a></td>
<td>New: Step.bat can stop the execution of the sequence and subsequent steps by returning exit-code 99 or 98. 99 causes an erronous halt, while 98 causes a halt without error.<br />
New: SKIP in replacement part of regular expression allows skipping entire lines.<br />
New: Windows 7 supported (Beta)</td>
</tr>
<tr>
<td>build 529 <a class="downloadlink" href="http://www.stefanbuehlmann.com/s/wp-content/plugins/download-monitor/download.php?id=3" title="Version529 downloaded 68 times" >Tester (68)</a></td>
<td>previous stable build</td>
</tr>
</tbody>
</table>
<h2><a name="Requirements">Requirements</a></h2>
<ul>
<li>Windows XP, NT, 2003, 95, Linux or Solaris 10</li>
<li><a href="http://java.sun.com/" target="_blank">Java 1.4</a> or higher</li>
<li><a href="http://jakarta.apache.org/commons/io/" target="_blank"> Apache Commons IO 1.0</a> (commons-io-1.0.jar &#8211; included in download)</li>
<li><a href="http://dbunit.sourceforge.net/" target="_blank">dbUnit 2.1</a> (dbunit-2.1.jar &#8211; included in download)</li>
<li><a href="http://www.junit.org/" target="_blank">jUnit 3.8.1</a> (junit.jar &#8211; included in download)</li>
<li>If you want to try the examples, you require one of the supported databases with a corresponding jdbc-driver. For example you can use:
<ul>
<li><a href="http://dev.mysql.com/downloads/" target="_blank">MySQL 4.1.11 or MySQL 5.0.18</a></li>
<li><a href="http://dev.mysql.com/downloads/" target="_blank">MySQL Connector/J 3.1.12</a> (for simplicity Tester comes with this jar  included.)</li>
</ul>
</li>
<li>You require a good diff-Tool to compare files and directories. I very much like <a href="http://www.componentsoftware.com/Products/CSDiff/" target="_blank">csDiff</a>, because it has a very intuitive way to compare .csv-files and entire directories. It shows differences within the line and not only differing lines.</li>
</ul>
<h2><a name="Databases">Tested Databases</a></h2>
<ul>
<li>MySQL 4.1.11, 5.0.18 (both with Connector/J 3.1.12)</li>
<li>Oracle 9i, 10</li>
<li>DB2 8.2 (not tested under Linux)</li>
</ul>
<h2><a name="Installation">Installation</a></h2>
<ul>
<li>Install the database of your choice. A default installation will do. Setup a schema and a user with full privileges on that schema. Have the corresponding jdbc-driver and the jdbc connect string ready.</li>
<li>Unzip the download to the directory of your choice.</li>
<li>Under Linux you must make the script files executable:<code><br />
chmod a+x All_Clean_make_Executable<br />
./All_Clean_make_Executable<br />
</code>This script applies chmod to all other scripts, which must be executable.</li>
<li>In script Tester.bat (or Tester under Linux) you should edit the following variables at installation time:
<ul style="margin-top: 0cm;" type="disc">
<li><a name="MYSQL_BIN">MYSQL_BIN</a>, <a name="DB2_BIN">DB2_BIN</a>, <a name="ORA_BIN">ORA_BIN</a>: These variables should point to the installation bin-directory of the commandline interface to your database, e.g. mysql or sqlplus. Tester.bat and the Step.bat scripts should not rely on having that directory on the path.</li>
<li><a name="JDBC_JAR">JDBC_JAR</a>: Make sure variable JDBC_JAR points to your jdbc-driver. You have to edit it, if you don&#8217;t use the standard MySQL-Driver supplied. JDBC_JAR is inserted into the <a href="#classpath">classpath</a>.</li>
<li><a name="LD_LIBRARY_PATH">LD_LIBRARY_PATH</a>: This variable is only required for Oracle under Linux, such that the jdbc driver finds the correct shared library.</li>
<li><a name="CONFIGDIR">CONFIGDIR</a>: Make sure variable CONFIGDIR points to the directory of your  <a href="#configurationfiles">configuration files</a>. You have to change it, if you don&#8217;t use the preconfigured <a href="#configs">configs</a> directory (inside directory Tester) or if you want to use the <a href="#ChapIndirection">indirection Feature</a>. A &#8220;;&#8221;-separated list of configuration directories is accepted. CONFIGDIR is inserted into the <a href="#classpath">classpath</a>.</li>
<li><a name="PROPFILES">PROPFILES</a>: Make sure variable PROPFILES lists your configuration .properties-files in directory <a href="#CONFIGDIR">CONFIGDIR</a>, if you don&#8217;t use the standard <a href="#configurationfiles">Configuration Files</a> supplied in directory <a href="#configs">configs</a>. PROPFILES corresponds to <a href="#Prop.propertyfiles">tester.propertyfiles</a>.</li>
</ul>
</li>
<li>All examples rely on the configurations defined in the properties files in directory <a href="#configs">configs</a> (properties are described in Chapter <a href="#ChapProperties">Properties / Parameters</a>). For the beginning, you should use these files and edit them as follows:
<ul style="margin-top: 0cm;" type="disc">
<li>In file <a href="#configs">configs</a>\<a href="#myStandard.properties">myStandard_*.properties</a> edit the values for my.jdbc.driver, my.jdbc.connection &amp; my.jdbc.schema.</li>
<li>In file <a href="#configs">configs</a>\<a href="#mySecured.properties">mySecured_*.properties</a>  set the values for my.jdbc.user and my.jdbc.password.</li>
</ul>
</li>
<li>Tester comes with five examples: <a href="#ChapBillingExample">Billing Example</a>, <a href="#Downloader">Downloader</a>, <a href="#Uploader">Uploader</a>, No_DB_Test &amp; Performance. Each of the examples comes with two start scripts (*_Tester.bat for Windows and *_Tester for Linux) and two scripts to clean up temporary data, produced during a run (Clean.bat for Windows and Clean for Linux). The first two of these scripts contain two variables, you have to edit:
<ul style="margin-top: 0cm;" type="disc">
<li><a href="#DB_TYPE">DB_TYPE</a>: Edit DB_TYPE to MYSQL, DB2 or ORACLE, depending on what your database is.</li>
<li><a name="TESTER_HOME">TESTER_HOME</a>: TESTER_HOME should contain the installation directory of Tester, namely, the directory, where Tester.bat is.</li>
</ul>
</li>
<li>Now your installation is complete. To test it, double-click on <a href="#Billing_Example_Tester.bat">Billing_Example_Tester.bat</a>. If the test finishes with the message &#8220;normal end&#8221;, then your installation is ok.</li>
<li>Since you might later change <a href="#ChapBillingExample">Billing Example</a>, <a href="#Downloader">Downloader</a> &amp; <a href="#Uploader">Uploader</a>, it makes sense to keep the downloaded zip and the completed configuration.</li>
<li>To go furhter from here, read the <a href="#Top">Documentation</a>.
</li>
</ul>
<p>
Note: There is a print link embedded within this post, please visit this post to print it.<br />
 </p>
]]></content:encoded>
			<wfw:commentRss>http://www.stefanbuehlmann.com/s/?feed=rss2&#038;p=186</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Absurde Technik</title>
		<link>http://www.stefanbuehlmann.com/s/?p=103</link>
		<comments>http://www.stefanbuehlmann.com/s/?p=103#comments</comments>
		<pubDate>Sat, 14 Mar 2009 15:31:04 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[HiFi media center blueray HD HDTV mp3 cablecom marian netgear philips logitech dennon Dipl. Ing. Fust AG]]></category>

		<guid isPermaLink="false">http://localhost/s/?p=103</guid>
		<description><![CDATA[<p>Musik, Film und Fernsehen sind schon längst digital geworden. Können PC, Fernsehen und Musik vernetzt werden? Oder ist das nur Theorie? Kann mein Fernseher Filme aus dem Internet zeigen? Und wie bringe ich meiner Stereoanlage Internetradio bei? Immerhin bin ich Informatiker und so wollte ich mal wieder den Puls der Zeit fühlen.</p> <p>Meine absurden [...]]]></description>
			<content:encoded><![CDATA[<p>Musik, Film und Fernsehen sind schon längst digital geworden. Können PC, Fernsehen und Musik vernetzt werden? Oder ist das nur Theorie? Kann mein Fernseher Filme aus dem Internet zeigen? Und wie bringe ich meiner Stereoanlage Internetradio bei? Immerhin bin ich Informatiker und so wollte ich mal wieder den Puls der Zeit fühlen.</p>
<p>Meine absurden Erlebnisse beim elektronischen Hochrüsten des Wohnzimmers zeigen den Stand der Technik, aber auch, dass das wahre Leben witziger ist als jede Komödie. Doch alles schön der Reihe nach.</p>
<h2>Meine Installation</h2>
<p>Im Sommer 2008 habe ich mein Wohnzimmer elektronisch aufgerüstet. So sieht meine neue Installation aus:</p>
<p>Die Geräte sind wie folgt verkabelt:</p>
<p><img style="width: 543px; height: 422px;" alt="Geräteverkabelung" title="Geräteverkabelung" src="wp-content/gallery/varia/Folie1.JPG"></p>
<p>Cablecom Wanddose: Von der Wanddose verwende ich zwei Stecker, einerseits den Antennenanschluss für den Cablecom HD Receiver und andererseits den Anschluss für das Modem.</p>
<p>Cablecom Modem: Das Modem bedient das Telefon und stellt den Internetzugang her.</p>
<p>Netgear Router: Der Netgear Router FVS338 ist Firewall und VPN-Endpunkt für den Support meiner Familie und Freunde.</p>
<p>Hewlett-Packard Media Center PC: Dabei handelt es sich um den HP Pavilion Elite m9270.ch Desktop-PC. Darin eingebaut findet man eine NVIDIA Grafikkarte GeForce 7900 GS mit HDMI-Ausgang (liefert ein HD-Signal), einen Blueray-Player, eine analoge TV-Karte und dazugekauft eine Marian Trace Alpha Soundkarte. Der PC hat Windows Vista Home Premium mit dem Windows Media Center installiert. Dazu gehört eine Fernsteuerung.</p>
<p>Cablecom HD Receiver: Der HD Receiver von Cablecom wurde in der Zwischenzeit durch den HD Recorder ersetzt.</p>
<p>Philipps HD-Fernseher 42PFL9603: Der Fernseher liefert ein sehr ansprechendes HD Bild. Er empfängt Signale vom Cablecom HD Receiver, vom PC oder von der Wanddose das althergebrachte analoge TV-Signal. Der Fernseher hängt auch am PC-Netzwerk und &#8211; zumindest theoretisch &#8211; am Internet.</p>
<p>Dennon analoger Verstärker DRA-935R: Ein inzwischen 18-jähriger analoger Verstärker mit integriertem analogem Radio-Tuner. Den Verstärker wollte ich nicht ersetzen, da er immer noch hervorragenden Musikgenuss gewährleistet, die Musikboxen hohen Ansprüchen genügen und weil der ganze Rest (Fernseher und PC) schon teuer genug waren. Der Verstärker empfängt die Audio-Signale des Fernsehers und des PCs.</p>
<p>Logitechs programmierbare Fernbedienung Harmony One: Die Harmony One ersetzt die Fernsteuerungen des Fernsehers, des HD Receivers, des PCs und des Verstärkers.</p>
<h2>Heute, da (fast) alles funktioniert, habe ich folgende Möglichkeiten</h2>
<p>Digitales Fernsehen in HD-Qualität über den HD Recorder von Cablecom oder von der Blueray-Disk des PCs. Mit HD gehen einem die Augen auf! Ohne HD fühle ich mich fast wie sehbehindert.</p>
<p>Digitales Fernsehen in Standard-Qualität: Über den Cablecom Recorder oder aus dem Internet, entweder mit Programmen wie Zattoo oder Nello oder &#8211; und das ziehe ich definitv vor &#8211; mit dem Windows Media Center. Speziell die Medienbibliotheken von ZDF und SF finde ich genial, so ziemlich das geilste was es gibt. Da kann ich deren Eigenproduktionen zu jeder beliebigen Zeit abrufen. Und die Bildqualität des Windows Media Centers ist beachtlich, fast so gut wie ein analoges Signal.</p>
<p>Analoges Fernsehen entweder direkt aus der Wanddose oder via die analoge TV-Tunerkarte im PC. Ich sehe jedoch nie mehr analog fern, wegen der Bildqualität und der eingeschränkten Senderauswahl.</p>
<p>Filme kann ich ab Blueray-Disk (HD-Qualität) oder ab DVD mit dem PC ansehen. Leider habe ich noch keinen HD-fähigen Video-OnDemand Anbieter im Internet gefunden, der auch die Schweiz bedient. Wenn Du einen kennst, sag&#8217;s mir bitte. Den OnDemand Service der Cablecom verwende ich nie, weil die Filmauswahl sehr beschränkt und auf ein eindeutig nicht-intellektuelles Publikum zugeschnitten ist. Und wenn ich mir mal einen HD-Videorecorder kaufe, kann ich den am PC oder direkt am Fernseher anschliessen.</p>
<p>Musik: Meine ganze CD Sammlung ist im PC gespeichert und inzwischen kaufe ich rege neue CDs indem ich diese vom Sofa aus direkt aus dem Internet runter lade. Ich könnte CDs auch mit dem PC abspielen, aber einfacher ist es, man lädt diese einmalig in den PC und spielt sie dann mit dem Media Center ab. Vor einer Party kann ich schnell mal 15 CDs zum Abspielen in die Warteschlange stellen und schon habe ich den ganzen Abend schöne Musik.</p>
<p>Digitales Radio bekomme ich via Cablecom Box und Fernseher oder mit dem Media Center aus dem Internet. Im Media Center verwende ich das Plug-In von Shout Cast, welches zig-tausend Sender zur Verfügung stellt. Zum Beispiel hören wir oft <a href="http://www.energy927fm.com/">http://www.energy927fm.com/</a>. Das erinnert uns an unsere USA Ferien im Sommer 20008. Das Media Center kann auch Musik aufnehmen und mit Titel- und Interpreteninformationen aus dem Internet versehen.</p>
<p>Analoges Radio hören kann ich mit dem mit dem analoger Verstärker von Dennon. Im Gegensatz zum Fernsehen, ist analoges Radio dem digitalen qualitativ immer noch ebenbürtig, wenn nicht gar überlegen. Einzig die Senderauswahl ist natürlich eingeschränkt.</p>
<p>Bilder: Meine Bildersammlung ist natürlich auch schon längst auf dem PC verfügbar. Und mit dem Media Center kann man eine &#8220;Dia-Show&#8221; auf einfachste Weise mit Musik hinterlegen. Ab und zu verwende ich den Fernseher wie einen Bilderrahmen.</p>
<p>Alle auf dem PC gespeicherten Bilder, Filme und Audio-Dateien kann ich auch von meinen anderen Computern im Haushalt abrufen. Speziell hilfreich um iPods und PDAs mit Musik zu bestücken. Auch der Fernseher kann darauf via Netzwerk direkt zugreifen. Diese Funktion verwende ich jedoch nie.</p>
<p>Fernsteuerung: Die Harmony One ist &#8211; einmal richtig konfiguriert &#8211; wirklich intelligent. Ein Knopfdruck reicht, um alle für ein bestimmtes Szenario benötigten Geräte ein und alle anderen aus zu schalten und um alle Signal-Ein- und -Ausgänge korrekt zu verbinden. Beispiel: Habe ich analoges Radio gehört und möchte nun mit der Cablecom Box fern sehen, reicht ein Knopfdruck um den Dennon Verstärker auszuschalten (fall ich denn den Ton nicht auch über diesen laufen lassen möchte), den Fernseher und die Cablecom Box ein zu schalten und dem Fernseher zu sagen, er soll das Bildsignal vom HDMI-Eingang 1 beziehen, wo die Cablecom Box angeschlossen ist. Möchte ich dann zum Media Center wechseln, lässt sie den Fernseher das Bildsignal vom HDMI-Eingang 2 beziehen, wo der PC dran hängt, schaltet sie den Verstärker wieder ein und lässt diesen das Audio-Signal vom PC beziehen und schaltet die Cablecom Box ab.</p>
<h2>Was die wunderbare Technik heute noch nicht schafft</h2>
<p>Den PC muss ich noch von Hand einschalten. Hintergrund: Die Fernsteuerung kann nur einen PC aus dem Stand-By einschalten aber meine Audio-Karte liefert in diesem Fall keinen Ton. Ich hoffe das kommende Service Pack 2 von Windows Vista löst dieses Problem endlich, sonst muss ich auf Windows 7 warten.</p>
<p>Sehr gerne würde ich das HD-Signal des Cablecom Recorders auch in den PC schleusen und dort aufnehmen und weiter verarbeiten wollen. Doch das ist wohl noch eine Frage von leistungsfähigerer PC Hardware? Und natürlich hat die Cablecom Box nur einen HD Ausgang und der PC keinen HD Eingang. So fragt man sich, wieso der HD Recorder einen Ethernet-Anschluss hat.</p>
<p>Generell ist der Cablecom Recorder funktional behindert. So können die Sender nicht nach eigenem Geschmack umsortiert werden und aufgenommene Filme können nicht geschnitten werden (zumindest habe ich diese Funktion nicht gefunden). Schade ist auch, dass die Bildqualität des HD Recorders deutlich hinter jener des HD Receivers zurück bleibt. Der Unterschied ist markant.</p>
<p>Der Fernseher kann zwar als Media Center Client auf Musik und Filme des PCs zugreifen, doch das ist eher stiefmütterlich und funktioniert nicht in allen Fällen. Auch ist mir nicht klar, wieso der Fernseher einen Netzwerkanschluss hat, aber keinen Browser und kein Email Programm anbietet. Selbst neuere Firmware kann er nicht direkt vom Internet herunter laden.</p>
<p>Funkschalter von Conrad Electronic: Eigentlich wollte ich alle Geräte per Fernsteuerung vom Stromnetz nehmen können, speziell weil der Cablecom HD Receiver auch im Standby Strom wie ein Weltmeister verbraucht &#8211; man könnte Spiegeleier darauf braten! Doch hat sich zerschlagen weil sich die bestellten Schalter schon nach wenigen Stunden für immer ausgeschaltet haben. Ich hab&#8217;s dann nicht mehr versucht. Immerhin hat Conrad Electronic die Schalter anstandslos zurück genommen. Die Schalter waren auch Funk-basiert und nicht Infrarot-gesteuert und somit nicht von der Harmony One steuerbar. Wenn mir jemand gute Infrarot-gesteuerte Schalter nennen kann, wäre mir das eine Hilfe! Ich würde auch gerne meinen WLAN-Router (oben nicht gezeichnet) so ein- und ausschalten.</p>
<p>Ich habe keine Satelliten-Schüssel aufgestellt und ich habe keine digitale Surround-Sound Musikanlage gekauft.</p>
<p>Generell wünschte ich mir, dass ich nicht mehr überlegen muss, ob ich ein Fernseh-  oder Videosignal analog, digital, von der Cablecom Box oder als Stream aus dem Internet beziehe, genau so wenig interessiert es mich, ob ein bestimmter Radiosender analog oder digital von der Cablecom Box, dem Dennon Tuner oder als Stream aus dem Netz kommt. Auch sollten die Konzepte der Medienbibiliotheken von ZDF uns SF weiter ausgebaut werden. Gleiches sollte auch für OnDemand Filme und CDs möglich sein.</p>
<p>Liebe Weltkonzerne, sputet Euch!</p>
<h2>Nun wird&#8217;s lustig&#8230;</h2>
<p>Nun aber endlich zu den angekündigten, höchst absurden Aspekten dieser ganzen Technik-Wundertüte, zu den eigenartigen Aussagen und Antworten der Angestellten der verschiedenen involvierten Firmen, und den unglaublichen Täuschmanövern dieser ach-so-klugen Firmen. Wir wissen es alle &#8211; in der Computer-Welt ist das reale Leben meist komischer als jede Satire. Hier folgt die Hitparade meiner Alpträume:</p>
<p><a title="Hewlett-Packard" href="http://www.hp.com/" target="_blank">Hewlett-Packard</a>: Da kaufe ich also einen Media Center PC von <a title="Hewlett-Packard" href="http://www.hp.com/" target="_blank">Hewlett-Packard</a>, einen PC der speziell für digitale Medien ausgerüstet sein soll, der unter anderem einen Blueray-Player und eine HD-fähige Grafikkarte umfasst.</p>
<ul>
<li>Aber ausgerechnet Blueray-Filme kann ich mit dem Media Center nicht abspielen. Hewlett-Packard legt dafür ein absolut untaugliches Anfängerprogramm bei, das weder mit der Fernsteuerung bedient werden kann, noch sonst was taugt. Ich muss selbst ein Plug-In kaufen um HD Material direkt aus dem Media Center zu schauen.</li>
<li>Verbindet man die analogen Audioausgänge der eingebauten Soundkarte mit dem Dennon Verstärker will man das Wohnzimmer fluchtartig verlassen. Ich habe noch selten so grauenhafte Musik gehört. Das Signal klirrt, brummt, scheppert und ist unerträglich. Mir bleibt nichts anderes übrig, als eine professionelle Audio-Karte (Marian Trace Alpha) dazu zu kaufen.</li>
<li>Auch will ich ja eigentlich Musik hören und nicht den PC. Liebe HP, baut doch bitte leise Media Center PCs.</li>
<li>Der PC hat zwar einen Standby Modus, doch verwendet man diesen, ist nach dem Wiedereinschalten der Ton weg. Das dürfte ein Problem von Microsoft sein, aber auch Marian hat keine wirkliche Antwort bereit.</li>
</ul>
<p>Der Fernseher von <a title="Philips" href="http://www.philips.de/" target="_blank">Philips </a>wurde mit einer veralteten Firmware geliefert. Die Dipl. Ing. Fust AG ist offenbar nicht in der Lage, sicher zu stellen, dass der Kunde aktuelle Produkte erhält. Aber das kann ja für einen Informatiker nicht wirklich ein Problem darstellen &#8211; so denke ich. Handbuch lesen, Website von Philipps besuchen, neue Firmware runter laden und &#8230; Ich hab&#8217;s nicht kapiert. Der Download enthält nicht die in der Beschreibung erwähnte Datei. So wende ich mich per Mail an den Support von Philipps. Nach einigem Hin und Her ist das Ergebnis: Philipps schickt mir &#8211; nach rund zwei Wochen &#8211; einen USB-Stick per altmodischer Post mit der notwendigen Software. Philipps ist nicht in der Lage mir einen funktionierenden Download per Web oder Email zu zu stellen. Dass der Fernseher auch am Netzwerk und somit am Internet angeschlossen ist, hilft nicht. Es gibt keine direkte Upgrade-Funktion.</p>
<p>Vor <a title="Logitech" href="http://www.logitech.com/" target="_blank">Logitech</a> hatte ich mal Respekt und ich war von der Qualität ihrer Produkte überzeugt, aber das ist nun definitiv vorbei. Die Harmony One ist zwar ein tolles Teil, wenn man es schlussendlich schafft sie zu konfigurieren. Aber das braucht ganz schön Nerven.</p>
<ul>
<li>Die Software ist unverständlich &#8211; ich würde gar sagen unverstehbar. Sehr viele Masken und man weis nie, wo man starten muss, um nach dem x-ten Schritt das Gewünschte konfigurieren zu können. Bei mir läuft so etwas unter dem Stichwort Pfui-GUI. Noch nie etwas von Drag-and-Drop gehört? Ich frag mich, wie lange Logitech ein Produkt verkaufen will, dass derart erklärungsbedürftig ist. Jedenfalls meine vielen Anrufe und Mails dürften deutlich höhere Kosten zur Folge gehabt haben als der Kaufpreis des Teils. Ich hab mich schon gewundert, wieso so viele Telefonnummern auf dem Beipackzettel waren.</li>
<li>Kommt hinzu, dass das Ding offenbar einen Wackelkontakt hat (sollte ein Bewegungsmelder sein) und daher ab und zu mal wieder durchstartet. </li>
<li>Aber das wohl lustigste war die Aussage eines &#8211; wohlverstanden sehr netten &#8211; Supportmitarbeiters. Also, ich kaufte diese Funkschalter welche nach kurzer Zeit den Geist aufgegeben haben. Bevor ich diese kaufte, schaute ich auf der Logitech Website nach, ob die Fernsteuerung diese Funkschalter unterstütz. Ja, tut sie. Nach Eingabe von Hersteller und Modell erhielt ich einen grünen Haken. Nach vielen erfolglosen Versuchen die Dinger mit der Fernsteuerung anzusprechen erklärt mir dieser Mitarbeiter, dass die Datenbank, welche man über die Website zur Kompatibilität abfragen kann, aus Einträgen entsteht, die Kunden machen, wenn Sie versuchen ihre Fernsteuerungen dafür zu konfigurieren. Ob diese Versuche jemals von Erfolg gekrönt waren, wird nicht geprüft. Es gibt keine Qualitätskontrolle dieser Datenbank durch Logitech. Der grüne Haken will wohl heissen, der Kunde war noch grün hinter den Ohren, oder was?</li>
</ul>
<p><a title="Dipl. Ing Fust AG" href="http://www.fust.ch/" target="_blank">Dipl. Ing. Fust AG</a>: Die obigen Produkte kaufte ich bei der Dipl. Ing. Fust AG. Na ja, das sind viele nette, gut aussehnde, aber unwissende Verkäufer. Verkäufer die zur Not auch mal Lügen? Wenn man fragt, ob das Produkt dies oder jenes kann, ist die Antwort grundsätzlich immer ja. Bei mündlichen Versprechen wird es schwierig diese einzufordern. Aber zum Glück lies ich mir den Verkaufsprospekt kopieren. Da stand, der PC hätte einen HDMI-Ausgang. Nach dem Auspacken stellte ich jedoch fest, dass dem nicht so ist. Erst versuchte man mich für dumm zu verkaufen und einen DVI-HDMI-Übergangsstecker anzudrehen. Erst als ich in der Dokumentation des Fernsehers schwarz auf weis zeigen konnte, dass damit ein Signalverlust einhergeht, lenkte man ein. Man tauschte mir die Grafik-Karte aus. Als ich dann mit dem PC wieder Zuhause ankam, stellte ich fest, dass ein Virus auf dem PC ist. Danke!</p>
<p><a title="Cablecom" href="http://www.cablecom.ch/" target="_blank">Cablecom</a> &#8211; ein Drama in 100&#8217;000 Akten. Wir wissen es ja alle, die Amis wollen die Schweizer ausnehmen.</p>
<ul>
<li>Eigentlich wollte ich ja einen HD Recorder von Anfang an, aber Cablecom hat rund ein Jahr Verspätung gehabt das Gewünschte zu liefern. Nachdem ich schon ein Jahr gewartet habe und der Recorder schon ein halbes Jahr verspätet war, bestellte ich also erst einmal den Receiver.</li>
<li>Aber immer wenn ich den HD Receiver frisch eingeschaltet hatte, dauerte es zwischen 10 und 20 Minuten, bis er sich geruhte auf die Fernbedienung zu reagieren. Danach ging es. Man muss also um 19.10 Uhr die Box einschalten, wenn man um 19.30 Uhr die Tagesschau sehen will. So rufe ich den berühmten Support von Cablecom an. Nicht nur einmal warte ich 30 und mehr Minuten, um anschliessend einen völlig überraschten Mitarbeiter zu sprechen, der nicht glauben kann was ich ihm erzähle. In letzter Verzweiflung bat man mich dies und jenes zu versuchen &#8211; jedes mal mit einer Pause von mindestens einem Tag, denn ich musste dann ja die Box erst wieder ausschalten und warten bis sie &#8220;abgekühlt&#8221; ist. Und wenn sie dann zum nächsten Versuch bereit ist, hies es ja erst wieder 20 Minuten warten und dann wieder 40 Minuten Musikgeklimper am Support-Nabel. Nach mehreren solchen Runden, jedes Mal mit einem neuen, ebenso ungläubigen Mitarbeiter, entschloss sich dieser, mir einen Boxentausch anzubieten. Es sei doch selbstverständlich, dass man mir eine kaputte Box austauschen würde. Das dauerte dann zwei Wochen, bis das Christkind kam. Doch &#8211; oh Schreck &#8211; die neue Box bestand ebenso auf 20 Minuten Aufwärmzeit! Völlig entnervt rief ich erneut die heisse Linie von der Kabel kommt an. Offenbar hatte das Schicksal nun Erbarmen mit mir, denn die Dame am anderen Ende meinte, es sei doch ein bekanntes Problemchen, dass dieser Infrarot-Empfänger etwas sensibel sei. Ich soll einfach so ein braunes Klebband vorne neben dem Display, vor den Infrarot-Empfänger kleben. Und wirklich, man glaubt es kaum: Das war die Lösung! Cablecom ist also weder in der Lage den Geräten einen Beipackzettel mit einem entsprechenden Hinweis dazu zu packen, noch wissen die vielen netten Mitarbeiter Bescheid. Sie finden auch nichts in Ihrer Supportdatenbank.</li>
<li>Nachdem das dann endlich funktionierte, konnte nach weiteren 6 Monaten Cablecom endlich den HD Recorder ankündigen. Sofort habe ich diesen bestellt. Nach einem Monat schrieb ich dann mal ein Mail, wo das Teil denn bliebe, ob es etwa Lieferengpässe gäbe? Mehrere solche Anfragen blieben unbeantwortet. Als ich es dann wieder über den Support-Nabel versuchte, wurde ich erst mal wieder von Pontius zu Pilatus verbunden, bis mir dann einer der Herren sagte, er löse meine Bestellung noch nochmals aus. Am Abend rief mich dieser Herr jedoch an und sagte, er hätte gesehen, dass diese erneute Bestellung zu einem Fehler in ihrem System geführt hätte. Er könne da nichts mehr tun, ich soll mich doch bitte am Montag an die nächste Telefonnummer wenden. Man bemerke, Cablecom Mitarbeiter können sich nicht gegenseitig eine Notiz zustellen &#8211; oder sie wissen, dass ihre Kollegen nicht lesen (können/dürfen). So ging es weiter und weiter und weiter und drei lange Monate später kam dann endlich diese Box. Ich wollte schon einen braunen Klebstreifen auf den Telefonhörer kleben (ist ja auch von Kabel kommt).</li>
</ul>
<p><a title="Netgear" href="http://www.netgear.de/" target="_blank">Netgear</a>: Weil ich ja jetzt etwas mehr Bandbreite brauche, um übers Internet fern zu sehen, habe ich bei Cablecom meinen Internetzugang auf 15&#8217;000 KBits/s erhöhen lassen. Dann stellte ich fest, dass mein alter Router da nicht mehr mitkommt. Da dieser von Netgear ist und ich eigentlich sehr gute Erfahrungen mit Netgear hatte, bestellte ich ein Nachfolgemodell. Zuerst wunderte ich mich, dass auch Netgear einen etwas umständlichen Support hat. Ich wurde gebeten mich an die Firmen-Hotline zu wenden, nur weil es sich um einen VPN-Router handelte. VPN verwende ich, um Freunden und Familie sicheren remote-Support bieten zu können. Das hat seit Jahren bestens funktioniert. Nun wollte ich also ein leistungsfähigeres Nachfolgemodell wieder mit den VPN-Routern meiner Freunde und meiner Mutter verbinden. Das ging nicht. Nach endlosem hin und her &#8211; ich zweifelte schon daran, ob das Produkt wirklich von der gleichen Firma stammt &#8211; fanden wir heraus, dass eines der Passworte nur Buchstaben und Ziffern, aber keine Sonderzeichen enthalten darf. Im Jahr 2008 darf man doch gar keine Passworte mehr verwenden, die nicht auch Sonderzeichen enthalten. Lebt Netgear hinter dem Mond? Bei VPN geht es doch um eine sicherheits-relevante Komponente!</p>
<p><a title="Marian" href="http://www.marian.de/" target="_blank">Marian</a>: Mit der Soundkarte Trace Alpha rettete mich Marian. Die einzige Firma im Verein, welche tadellosen Service bot. Nur leider warte ich immer noch auf den Treiber-Update, welcher den Powersafemodus unterstützt.</p>
<p>So, das war&#8217;s für den Moment. Vor all den weiteren kleinen lästigen Details von nicht funktionierender Software will ich Euch verschonen.</p>
Note: There is a print link embedded within this post, please visit this post to print it.
]]></content:encoded>
			<wfw:commentRss>http://www.stefanbuehlmann.com/s/?feed=rss2&#038;p=103</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

