Hypothesized relations between variables are what we call models. Models are at the core of prediction science.
This article reviews different types of models, compares vocabulary across disciplines and how to build and interpret models.
Most models are built as black boxes, while some are structural.
Models can also be categorized as deterministic or probabilistic.
General black box model: (input - blackbox - output)
\[ x \rightarrow [f] \rightarrow y \]
General black box model (two parts): (Observables are stochastically related through a joint distribution.)
\[ X \sim f(X) \] \[ X \rightarrow f(Y|X) \rightarrow Y \]
Reduced form 1: Moment description
First-order description:
\[ X \rightarrow [f] \rightarrow E(Y|X) \]
Note: This is the view adopted in econometrics.
Second-order description: (Electrical engineering, w.s.s.)
Reduced form 2: (Bayesian) Network
\[ X ~ \prod_{i=1}^{n} f( X_i | \Pi_{X_i} ) \]
\( \Pi_{X_i} \) denotes the parent nodes of \( X_i \) in Bayesian network \( G={X,E} \).
Different fields and theories have different terminology for their models, but they share the same essence as outlined above. Collecting these concepts is the first step towards a general theory of prediction science. [Interestingly, this saying is rarely used by people other than my adviser.]
constitutive relations [physics]
mapping
system [electrical engineering]
input-output [economics]
Other names for black box models:
structural models [econometrics]
feedback loop [cybernetics]
regulatory circuits [molecular biology]
idealized model [physics]
program [computer science]
Other names for structural models:
Point: the building block of mathematical theories.
Quantitative Theory Categories
Information used: prior knowledge about the system, measurements on variables, etc.
Impossibility principle of modeling building: It is impossible to build a model that provides meaningful output at a fine scale, with most observables at a coarser scale and only limited information that the fine scale.
Building structural models: hypothesization, abstraction, simplification.
普朗克推导黑体辐射定律时，做了能量子假设，使得导出的定律与全频谱的实验数据吻合得很好。相比而言，此前的维恩公式仅仅适用于低频部分。【经典建模实例】
Goodness-of-fit measures [statistics]
Occam's razor (principle of parsimony)
Interpretation of parameters:
ceteris paribus: Should the traditional "all else being equal" be replaced by a more practical "all else left unknown", conclusions can become much more useful. This symbolizes a transition from deterministic differential viewpoint to statistical presentation.
Topics in modeling: