This paper draws distinctions among various concepts related to tipping points, robustness, path dependence, and other properties of system dynamics. For each concept a formal definition is provided that utilizes Markov model representations of systems. We start with the basic features of Markov models and definitions of the foundational concepts of system dynamics. Then various tipping point-related concepts are described, defined, and illustrated with a simplified graphical example in the form of a stylized state transition diagram. The tipping point definitions are then used as a springboard to describe, formally define, and illustrate many distinct concepts collectively referred to as "robustness". The final definitional section explores concepts of path sensitivity and how they can be revealed in Markov models. The definitions provided are presented using probability theory; in addition, each measure has an associated algorithm using matrix operations (excluded from current draft). Finally an extensive future work section indicates many directions this research can branch into and which methodological, conceptual, and practical benefits can be realized through this suite of techniques.