5/1/2008 Author: Panos Y. Papalambros
Volume: 130(5) – May, 2008
Replication of previous results is a basic element in the advancement of science. Traditionally, one of the tenets in publishing research work of archival value is the ability of the reader to replicate the results described in the paper. The implication is that the authors must provide sufficient detail to allow the reader to attempt such replication. Theoretical results can be replicated if the assumptions and basic steps in a proof are properly given. Experimental results can be replicated if the conditions of the experiment are fully described. Computational results can be replicated if sufficient details are given for the computational model (simulation), computing environment, algorithm, and other programming parameters.
As the complexity of research work reported has increased, the available publication space (literally, journal pages) has been inadequate for accommodating the amount of detail necessary to allow results replication, as we might have expected it traditionally. In many cases involving computation, it would be unrealistic to expect to have sufficient details or representation in printed form. Thus, in many cases, we trust the authors’ reported work, and if we want to replicate it we must contact authors directly, exchange codes or ask further clarifications.
In a similar vain, the traditional “test problems” have become increasingly sparse. In many areas, a new idea, theory or algorithm are often demonstrated or tested on relatively simple problems, previously solved by other methods. The advantages of the new approach are then shown in comparison to previous ones, or at least the methods are clarified on a relatively well-known problem. Here again, as our ability to solve complex problems increases, so does the complexity of the test or demonstration problems; and thus our ability to report them sufficiently for replication decreases.
An obvious way to address this situation is to take advantage of digital publication, which removes many of the obstacles of space and representation. In January 2008 ASME launched the ASME Digital Library www.asmedl.org as its primary repository of current and archival literature featuring Transaction Journals from 1990 to the present, Conference Proceedings from 2002 to the present, and ASME Press E-Books selected from 1999 to the present. As the ASME Digital Library continues to expand, it ultimately will include the complete archive of ASME’s Transactions Journals dating back to 1880. In addition, JMD has for some time now maintained an “alternate” site, to allow posting of upcoming article titles, thus enabling more timely citation of JMD articles soon to appear.
In view of the above, I have recently procured the domain name www.asmejmd.org to serve at least two goals: (i) to continue the advanced posting of upcoming publications; and (ii) to provide a venue for authors of JMD articles to post the additional details that would allow replication of results contained in their published article. Professor Matt Parkinson from Penn State University has agreed to perform editorial duties for this new site. We plan this effort as a complement to ASME’s Digital Library, and perhaps in the future the materials in question will be incorporated there. For now, the new site will serve as a bona fide archival source of materials that are too extensive or better suited to a digital rather than printed presentation. As we develop the site, I would welcome our readers’ and authors’ input with respect to its content and function. I would also welcome any other thoughts you may have in addressing the issue of results replication and publication of test problems.
