""Battle Of Engineers"" Mars DTV Test Results

By George Leopold And Junko Yoshida, EE Times

2-May-00

US broadcasters hope to resolve a long-running dispute over conflicting interpretations of digital TV engineering test data when an industry group begins side-by-side testing of competing modulation technologies as early as this month.The DTV tests, by Maximum Service TV (MSTV), a Washington, DC-based group representing local TV stations, are being planned as concerns grow among broadcasters and regulators about a lack of unbiased engineering analysis of test data on issues ranging from digital TV to low-power FM radio. Indeed, the conflicting interpretation of test data is setting up a ""battle of the engineers,"" Federal Communications Commission Chairman William Kennard recently told the National Association of Broadcasters (NAB). Kennard's concerns were echoed by the FCC's Michael Powell, who told broadcasters the rise in conflicting data is making it harder for regulators to render decisions on the high-stakes technical issues put before them. ""Neither side in [a technical debate] is infallible,"" Powell said. A recent controversy over an FCC decision to allocate broadcast spectrum for low-power FM radio stations - a move broadcasters said will cause unacceptable interference with their signals - prompted warnings about the manipulation of test data. But the issue has grown to mammoth proportions with the debate over test results for competing DTV modulation schemes. In the last several months alone, chip vendors, system companies and broadcasters have conducted DTV-transmission laboratory and field tests to measure the efficacy of the Advanced Television Systems Committee's 8VSB (vestigial sideband) digital TV modulation scheme. But the conclusions reported for the various engineering tests differed widely. On one side, Sinclair Broadcast Group, which is advocating an alternative modulation technology, asserted that ""serious problems exist with ATSC 8VSB indoor reception in strong signal environments."" But CBS came to the opposite conclusion after reviewing the results of DTV field tests in Philadelphia, Pennsylvania. ""It is evident that the current ATSC system is replicating NTSC reception coverages for both indoor and outdoor reception,"" network technicians reported. Meanwhile, Japan is gearing up for the launch of satellite-based digital TV service this fall. The service has been in field trials since November 1998 using a variant technique, called segmented COFDM, that it says is superior because it allows the flexible use of many different modulation schemes. Exhaustive but highly controversial tests in Brazil advocated for COFDM in general. The Brazilians were the first to conduct side-by-side comparisons of four modulation schemes adopted by three international DTV standards: ATSC's 8VSB, the 2k coded orthogonal frequency division multiplexing (COFDM) and 8k COFDM developed for Europe's Digital Video Broadcast (DVB), and the 4k COFDM of Japan's Terrestrial Integrated Services Digital Broadcasting (ISDB-T). The test report concluded that Anatel, Brazil's national telecommunications regulatory agency, should drop 8VSB in favor of COFDM. As more DTV test results emerge from various sources, broadcast engineers, system developers and chip designers face the formidable job of wading through data and analyzing the value of the engineering results. Given the contradictory data and conflicting interpretations, regulators and some industry executives wonder whether political demands are forcing rushed science and sloppy testing. Worse, they fear that engineers are turning into willing partners-in-crime, doing whatever it takes to ""interpret"" data in ways that promote their own political agendas while operating under the guise of science. Engineers and their defenders insist, however, that the incompatible test results aren't politically motivated. Rather, they cite poor methodologies or simple, subjective disagreement among the experts on what constitutes successful DTV reception. All sides in the debate, including such leading broadcasters as NBC and PBS as well as the ATSC, hope the MSTV tests, to be held at NBC's Washington affiliate, will clear the air. ""We'd like to do [the testing] quickly, but we want to do it cautiously"" to fill in all the gaps in the testing record, said MSTV's Victor Tawil. While critics said earlier tests used faulty methodologies that may have produced questionable results, Tawil said MSTV plans no changes beyond collecting ""traditional measurements"" of signal reception. ""In the digital world it's pretty clear: either the picture is there or it's not there,"" he said. Still, given manufacturers' concerns about indoor DTV reception, observers said MSTV will be under pressure to come up with firm test data to resolve the modulation dispute. Separate Testing Tawil confirmed that the group has had discussions with Japanese set manufacturers about future, separate testing of ISDB-T 4k COFDM systems. The start of separate tests would depend on how soon MSTV received equipment, Tawil and other industry executives said. NBC will supervise a COFDM task force during the MSTV tests. The network said it hopes the MSTV tests will reveal the ""differentials in performance between 8VSB and COFDM, especially in the areas of indoor reception and portable services."" NBC also said it is interested in the Japanese COFDM system. ""ISDB-T is a very interesting COFDM format because it is designed with a time-interleaving feature,"" said Jim Albro, NBC's Director of Transmission Engineering. Time interleaving has yielded superior performance in impulse noise tests compared with both 8k COFDM and ATSC's 8VSB, he said. Still, different observers place different weights on each test result, he said. ""Limited time for testing and limited access to the latest [receiver] equipment"" could be big factors in a test's outcome, Albro said. Despite MSTV's reassurances about testing procedures, observers said the issue might be inconsistent testing methodologies in the RF world. RF transmissions are affected by such variables as the terrain, the weather and even passing aircraft or pedestrians. It is difficult to replicate an RF field test and virtually impossible to get reproducible results, said Matt Miller, President and Chief Executive of NxtWave, a developer of modulation chips. ""If you carefully pick a location and run the same test ten times, you could easily get ten different results."" ATSC has set up a task force to look at the performance of RF systems, including receivers. Issues yet to be sorted out, however, include such basics as designing a solid testing methodology, agreeing on measurements and on a definition for ""successful"" reception, and achieving standards for objective testing. ""Get To The Bottom"" ""Our task force will get to the bottom of this,"" promised ATSC Chairman Robert Graves. Rather than condemn field tests as self-serving, many engineers remain believers in further testing and trust their own ability to evaluate the techniques and results accordingly. Broadcast engineers thus remain hopeful the MSTV tests will break the logjam. NBC, for example, had planned to conduct its own field testing but decided to join the MSTV tests in the interest of achieving consensus. ""With new tests conducted by a broad-based consortium, we hope to get much better access to the latest equipment,"" NBC's Albro said. Graves said he thinks ""MSTV will get the help they need to ensure that the tests are conducted carefully and appropriately."" Brazil's Society of TV engineers (SET) and Brazilian Association of Radio and Television Broadcasting Stations (ABERT), the bodies that had been authorized by Anatel to conduct DTV tests in that country, briefed more than 300 engineers on the results at the recent NAB convention in Las Vegas. While most welcomed the groups' exhaustive data, some claimed the Brazilians had conducted good testing but had based it on bad science. For example, ""data from spot-check sites was intermingled with data from a statistically meaningful set,"" said Wayne Luplow, Vice President of Standards and HDTV Promotion at Zenith Corp., the developer of 8VSB technology. While the Brazilians had tested the 8VSB and DVB 2k systems at nearly 130 field test sites, tests for ISDB-4k and DVB 8k were conducted at only a few selected sites that proved inhospitable to 8VSB. The Brazilians had compared apples and oranges, Luplow said, by mixing data collected in such differing ways and then assuming that the systems would function properly at other sites. ""This isn't how you get scientific results,"" he said. The ATSC filed extensive technical comments on the report with Anatel, stressing that 8VSB won the lab tests and that 8VSB also won the only statistically meaningful field tests. Underscoring the politics in the controversy, ATSC's Graves said, ""We have some concern now that SET/ABERT has taken a strong advocacy position in defense of their earlier conclusion to forsake VSB."" Gerry Kaufhold, a digital TV analyst with In-Stat, also suggested that ABERT/SET tests might have been predetermined to support COFDM. Kaufhold noted that Fernando Bittencourt, Director of Engineering at broadcaster Rede Globo, had said at the NAB briefing that ""in Brazil, we want to make sure that our local TV stations can provide features and services that differentiate from what cable TV has to offer, and that means mobile data delivery service."" Kaufhold wondered why Brazilians conducted no tests on mobile data services, if mobile applications are indeed the reason for their endorsement of COFDM, which is reputed to be strong in data delivery. ""Open And Fair"" Not everybody was critical of the Brazilian tests. Osamu Yamada, Director General of the Science and Technical Research Laboratories of NHK, Japan's national broadcasting corporation, called the tests ""open and fair."" Yamada participated in the NAB session as a representative of the Digital Broadcasting Experts Group, the Japanese body promoting ISDB-T. ""It's not a surprise to us that ISDB-T received the highest score"" in the Brazilian tests, Yamada said. Similar comparison tests held in Singapore and Hong Kong had similarly revealed ISDB-T to be the best-performing format, he claimed. But those test results have not been made openly available. NBC's Albro was among those who defended the Brazilian tests. ""I don't think the ATSC's analysis [of the tests] is really valid,"" he said. The Brazilians found difficulties with 8VSB that might be ""difficult to quantify in tables,"" Albro said. Criticizing their results based on the statistics shown in the tables alone is ""a pretty arrogant thing to do."" Asked about CBS' digital TV field test results in Philadelphia, ATSC's Graves gave CBS high marks for its technique. ""I find the work that CBS did to ensure that its transmitter system was functioning properly to be highly relevant and highly pertinent in explaining why CBS might have gotten substantially better results in Philadelphia than those obtained by others,"" Graves said. However, NBC's Albro wasn't so sure: ""I think there is an issue of usability to specsmanship."" Additional reporting by Yoshiko Hara.