UP | HOME
../../

Software Comprehension

Table of Contents

Abstract: What does it take to understand the source code of [large] programs that you did not participate in? Do you have the src code? Do you have its RSDI-docs?

1 Collected Quotes

  1. "Here is a not uncommon scenario in many workplaces. A neophyte […] programmer is assigned to maintain, debug, or enhance an application. The atmosphere is sink or swim, the system is complex, the code is sophisticated, the documentation is scant, and the programmer is bewildered. Questions slowly take shape. What, exactly, am I supposed to do? What part(s) of the application need my attention? Will a change to program X affect program Y? And, most critically, where do I start? What the poor programmer needs is a strategy for comprehending the program, then finding the sweet spots in the code as efficiently as possible." [from http://www2.sas.com/proceedings/sugi27/p068-27.pdf ]
  2. "Legacy systems have one or several of the following attributes: they were implemented many years ago, their technology became obsolete, their structure deteriorated, they represent a large investment, they contain business rules not recorded elsewhere, they cannot be easily replaced, or the original authors are not available. Software comprehension typically consumes more than a half of the difficult effort of maintaining legacy systems." [from Rajlich, ICSE, 1997]
  3. "I have never inherited a codebase I liked." Anonymous Developer.

2 Comprehension How To

  1. What are the (expected) givens?
    1. Source code files
    2. RSDI-docs
      1. Requirements Document
      2. Specs Document
      3. Design Document
      4. Implementation Document
      5. Testing Document
    3. Even in the best-run projects, any thing more than source code files, is a bonanza. Contribute such examples!
  2. Learn enough about the domain of the software:
    1. E.g., Compilers. You cannot expect to understand a compiler project without understanding parsing, code generation, etc. literature.
    2. This is true even for simple things, e.g., Tic-Tac-Toe.
  3. Browse the Empirical Software Engineering journal, special issue on Program Comprehension, Volume 18, Issue 2, April 2013 http://link.springer.com/journal/10664/18/2/page/1 Read the Preface.
  4. Generate "Design Documents"
    1. using whatever tools, but must be "dependable" (produce usable info)
    2. inserting prose annotations
    3. progressively improving prose annotations into verifiable/executable assertions
  5. Recall that assertions can be written as executable boolean methods in the implementation language without side-effects.
  6. Code refactoring
    1. Often done to "improve"
    2. Here do it to "understand", by breaking into pieces

3 Browsing Tools

  1. Browsing == Code Inspection
  2. Stand alone (tiny) tools: {ctags, etags, javadoc, …}
  3. http://doxygen.org Open source documentation system for software written in C++, C#, Java, Python, IDL, C and more. Can generate Class relationship diagrams and file relationships.
  4. https://www.google.com/search?q=static-analysis-plugins-for-intellij
  5. https://www.google.com/search?q=static-analysis-plugins-for-eclipse
  6. Source Navigator can display relationships between classes, functions, members, and display call trees mapping unknown source code for enhancement or maintenance tasks. For C/C++. 2014. Development stopped? http://sourcenav.sourceforge.net/
  7. 1 http://lxr.linux.no/ 2 https://elixir.bootlin.com/linux/latest/source 3 https://code.woboq.org/linux/linux/ Linux source code browsers
  8. https://www.gnu.org/software/global/links.html 'Source code reading' related sites
  9. Commercial Tools: JArchitect, NDepend, … [Free Trials?][Search for links]
  10. P. Anderson; M. Zarins, The CodeSurfer Software Understanding Platform Program Comprehension, 2005. IWPC 2005. Proceedings. 13th International Workshop on (January 2005), 2005, pg. 147-148. Reference. http://www.grammatech.com/research/technologies/codesurfer [Commercial; free trial]

4 Reverse Engineering

  1. Reversing binary files is termed Reverse Code Engineering, or RCE. Often used in malware analysis.
  2. Obfuscation is used to deter both reverse engineering and re-engineering.
  3. Canfora, et al., see Refs. Required Reading
  4. Tool: IDA https://www.hex-rays.com/products/ida/ Runs on Linux, Mac OS X, or Windows. "IDA has become the de-facto standard for the analysis of hostile code, …" [Commercial; free download and trial.] [free educational licenses]
  5. Tool: Ghidra https://github.com/NationalSecurityAgency/ghidra NSA Ghidra Software Reverse Engineering Framework, 2020 active FOSS. Runs on Linux, Mac OS X, or Windows.

5 Design Extraction

  1. Extracting design details from src code.
  2. Source code is reverse-engineered back to … design … specs.

6 Operations on Source Code

  1. Program Slicing: What could have affected this variable’s value? A program slice with respect to a given variable, v, is a set of variables the values of which can influence that of v. https://en.wikipedia.org/wiki/Program_slicing
  2. Ripple analysis: If a given statement is modified, where does it affect? And, how?
  3. Mutation. http://sites.utexas.edu/august/files/2020/08/ASEDEMO2018.pdf SRCIROR: A Toolset for Mutation Testing of C Source Code and LLVM Intermediate Representation

7 Obfuscation

  1. [dictionary def] Obfuscate: tr.v. -cated, -cating, -cates.
    1. To render obscure.
    2. To darken.
    3. To confuse: his emotions obfuscated his judgment. [LLat. obfuscare, to darken : ob(intensive) + Lat. fuscare, to darken < fuscus, dark.] -obfuscation n. obfuscatory adj
  2. There are companies that practice obfuscation to thwart reverse engineering.
  3. http://www.ioccc.org/ The International Obfuscated C Code Contest
    1. To write the most Obscure/Obfuscated C program within the rules.
    2. To show the importance of programming style, in an ironic way.
    3. To stress C compilers with unusual code.
    4. To illustrate some of the subtleties of the C language.
    5. To provide a safe forum for poor C code. :-)
    6. Source code of winning programs is included.
    7. 27th Contest happened in 2020 http://www.ioccc.org/2020/whowon.html
  4. Google search for java bytecode obfuscator

8 Good/ Responsible Software Development

  1. To help future readers of your software.
  2. Design-by-Contract
  3. Standard Literate Programming tools on Linux: cweb, cweave, ctangle; noweb; …
  4. http://www.ssw.uni-linz.ac.at/Research/Projects/RevLitProg/ " a system which allows … selective browsing. Zoom in at interesting points or jump to other locations according to control flow or other semantic relationships. This is the approach of hypertext. … Reverse Literate Programming …"
  5. https://dzone.com/articles/literate-programming-life Literate Programming Life Cycle 2010

9 Books on (Well-Documented) Specific Programs

  1. Knuth's Computers & Typesetting, https://www-cs-faculty.stanford.edu/~knuth/abcde.html TeX: The Program, and Metafont. The .web files of these can be legitimately downloaded. [PDF http://visualmatheditor.equatheque.net/doc/texbook.pdf legit?]
  2. Tanenbaum's example OS Design: Minix; Operating Systems Design and Implementation (3rd Edition), ISBN-13: 978-0131429383
  3. Operating Systems: Principles and Practice, Second Edition, by Thomas Anderson and Michael Dahlin. https://xinu.cs.purdue.edu/
  4. Jim Welsh and Atholl Hay: A Model Implementation of Standard Pascal, Prentice Hall, 1986
  5. Niklaus Wirth, et al. http://www.projectoberon.com/ "Project Oberon is a design for a complete desktop computer system from scratch. … Project Oberon: The Design of an Operating System, a Compiler, and a Computer – written by the designers, Niklaus Wirth and Jürg Gutknecht." https://people.inf.ethz.ch/wirth/

10 Reading List

  1. Many refs are embedded in the above.
  2. http://www.program-comprehension.org/ Recommended Visit. For awareness.
  3. Marouane Kessentini, W. Kessentini, H. Sahraoui, M. Boukadoum and A. Ouni, "Design Defects Detection and Correction by Example," 2011 IEEE 19th International Conference on Program Comprehension, Kingston, ON, 2011, pp. 81-90, doi: 10.1109/ICPC.2011.22. Required Reading.
  4. Russell Wood, "Assisted Software Comprehension", A Project Report, June 2003; {Is this a BS/ MS thesis from Imperial College London?} Reference.
  5. Gerardo Canfora, Massimiliano Di Penta, Luigi Cerulo, "Software Reverse Engineering: Achievements and Challenges", Communications of the ACM, Volume 54 Issue 4, Pages 142-151, 2011. Required Reading.
Copyright © 2020 • www.wright.edu/~pmateti 2020-09-27