N242-091 TITLE: An Open-Source Academic Publication Platform Tailored Toward Future Open Science Communications

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Advanced Computing and Software

OBJECTIVE: Develop an open software infrastructure to support modernized scientific and technical communication. The end product will support a variety of publishing goals, including flexible options to edit, review, comment, and compare related written works and be able to curate publications, proposals, abstracts, presentations, data, algorithms, and other communications.

DESCRIPTION: The current peer-review model for scientific literature publication is outdated. The review process is opaque to a fault, and the floor for what constitutes publishable content grows higher by the day. Many people and topics get lost in this mire; including good research hindered by incompatible, overly demanding or biased reviewers. At the same, platforms like Slack and Teams provide teaming tools; yet they are over developed for information distribution and lack an organized peer review process. A middle ground needs attention for dynamic content below journals but above more stovepiped information systems. While the proliferation of new open-source journals provide a simulacra of legacy journals, there exists a vacuum in open best practices and software methods to improve science communications at large. For example, how does one distribute well-reasoned but ultimately doomed research that tells a compelling and cautionary tale? Not only is the future of open science expanding via government mandate (OSTP 2022), there exists a need to track, catalog, archive, and otherwise compare technical work that would otherwise fall victim to the publish-or-perish mentality and have their contributions lost to the sands of time. The academic publication process seems ripe for disruption.

This SBIR topic seeks commercial innovations that can take advantage of new technologies in open software development cycle, version controlling, living documentation, continuous integration/continuous delivery, and others that have the potential to vastly change the paradigm of science/technical publication and communication to allow creation, curation, and distribution of knowledge in innovative ways. This may be achievable via development of a new software platform with a document and data store that better accounts for the variety of new communication methods, archival standards, and machine learning of key words/content to better serve all scientific work of relevance to the academic and government community, not just the flashiest success papers. Ultimately, this topic, seeks the development of both an open publication, open review platform and series of editorial standards that could be applied to a new journal-like medium for accepting a broader array of scientific communication such as null results in the geosciences.

PHASE I: The majority of this effort should focus on a survey of current geoscience publication methods and configurations, an assessment of expected future US government open science standards and procedures, and a proof-of-concept system level architecture of software components and processes for a modernized engine to support new science communication publication methods.

In the survey of current geoscience publications, the performer is expected to examine a series of editorial standards and protocol following popular journals and organizations such as from Nature/Springer, Elsevier, Wiley, the American Geophysical Union (AGU) or American Meteorological Society (AMS). Standards include document formatting, citation style, determining what constitutes a publishable unit, and a process for assigning peer-reviewers to topics (e.g., author-suggested reviewers, a roster of volunteer reviewers, etc.) that encourages repeat participation and ensures proper assignment of Subject Matter Experts (SMEs).) Reviewing open science mandates by the White House Office of Science and Technology and planned implementation strategies such as from NASA Open Science and NOAA will be compared to the previous publishing paradigms and used to contrast new needed publication functionality.

Outline front- and backend infrastructures for an open access hosting and open peer-review system with modern UI/UX for both desktop and mobile experiences. Considerations should be made for long term retention of content and scalability. Emphasis on lightweight, open-source and cloud-oriented solutions are preferred. The peer review system should include options for single-, double- and triple-blind reviews as well as fully open. Provide an open review option using a GitHub or Jira-like interface. For frontend planning, accessibility, including colorblind considerations and compatibility with popular mobile and desktop screen reading software (e.g., JAWS, VOX, TalkBack, etc.) are a priority. The design should also incorporate functionality for continued review/dialog as a living document, citation management and interoperability, and machine learning methods to suggest related work via key word and journal content analysis.

An outcome of the six-month Base effort should be a final report of background, anticipated functionality, technical challenges for software development and implementation, and recommendations for prototype development.

PHASE II: Develop, iterate, and prototype the software outlined in Phase I with an option period expanding the functionality and/or interacting with Naval research, university, and science publication partners for demonstration. End-to-end tests need to be conducted of multiple submission, review, and communication processes to ensure seamless operation for users. Emphasis should be on 1) replication of current journal publication standards, 2) demonstration of functionality that conforms to open science standards (such as tracking of review comments, replies, data, algorithms, and discussion toward living document type updates), and 3) extensibility to broader technical and science communication use cases, such as proposal reviews, special collection and discussion boards, public comment solicitations, and curation of historical documents (abstracts, preprints, conference proceedings, oral/poster presentations, etc.). Of particular interest is leveraging the developed infrastructure and metadata creation for advanced machine learning methods to better find and serve specialized related articles. Demonstrate such a capability using a large publication sample to find multiple, specific select groups of related topics.

PHASE III DUAL USE APPLICATIONS: Participate in local demonstrations for Office of Naval Research proposal tracking and review, special publications of technical reporting at a controlled level, and partnership with a geoscience publication entity (profit or non-profit) to demonstrate functionality via a new journal solicitation. Provide technical and editorial support to submitting authors and reviewers; stress testing of the system with metadata and other associations; and ingesting archived/historical publications for database indexing and analysis. Beyond Naval research use, dual-use commercialization is expected to be similar with other governmental entities with varying needs for software capabilities and data archival/analysis. Given the goal of open source architecture, it is anticipated that the cost model will involve varying degrees of user support, new functionality development, and other SaaS sustainment to align with vendor monetization goals.

REFERENCES:

1. Nelson, A. "Ensuring Free, Immediate, and Equitable Access to Federally Funded Research." Memorandum for the Heads of Executive Departments and Agencies, August 25, 2022. https://www.whitehouse.gov/wp-content/uploads/2022/08/08-2022-OSTP-Public-Access-Memo.pdf
2. Stall, S.; Bilder, G.; Cannon, M. et al. "Journal Production Guidance for Software and Data Citations." Scientific Data, 10, Article Number: 656 (2023). https://www.nature.com/articles/s41597-023-02491-7
3. Brezis, E. S. and Birukou, A. "Arbitrariness in the peer review process." Scientometrics, 123(1), 2020, pp. 393-411. https://link.springer.com/article/10.1007/s11192-020-03348-1#:~:text=The%20peer%20review%20process%20leads%20to%20arbitrariness%3A%20For%20the%20same,results%20of%20the%20NIPS%20experiment.
4. DellaVigna, S.; Pope, D. and Vivalt, E. "Predict science to improve science." Science, Vol. 366, Issue 6464, 25 October 2019, pp. 428-429. https://www.science.org/doi/10.1126/science.aaz1704
5. Haffar, S.; Bazerbachi, F. and Murad, M. H. "Peer review bias: a critical review." Mayo Clinic Proceedings, Vol. 94, No. 4, April 2019, pp. 670-676. https://www.mayoclinicproceedings.org/article/S0025-6196(18)30707-9/fulltext
6. Munafò, M. and Neill, J. "Null is beautiful: On the importance of publishing null results." Journal of Psychopharmacology, Vol. 30, Issue 7, p. 585. https://journals.sagepub.com/doi/full/10.1177/0269881116638813

KEYWORDS: Open science; open source, publishing; science writing; technical writing; curation; machine learning; archival; peer review

TPOC-1: Joshua Cossuth

Email: [email protected]

 

TPOC-2: Christopher Selman 

Email: [email protected]

---