Monday, November 20, 2006

Objectives of data analysis

Why analyse data?

you try to answer the question - is there a hidden determinism in your data?
and after knowing that you would like to
a) predict or
b) extract a deterministic signal from noisy background
c) gain better insight and understanding of the underlying dynamics

paraphrased from Chaos and Time-series analysis by Sprott.
(a similar thing is mentioned in kingston - see.)

My ideas -
Step 1:
Is there a hidden determinism in the data?
Traditional statistical technique -
Autocorrelation?

Itertative NN technique -
Test existence of the relationship using alternating division in data and using a large test set.

Step 2:
Gain better understanding of underlying dynamics:
is there a periodicity?
is there relationships between parameters?

Traditional statistical technique -
Periodicity -
Fourier analysis
Lyuponov exponents?

Relationships between parameters –
descriptive statistics – scatter diagram
ANOVA / MANOVA/

Iterative NN technique -
Periodicity –
??

Relationship between parameters -
Non – linear principal component analysis?

Sensitivity analysis
weights method shows that parameters are highly dependent
derivatives method shows that chlorophyll is more sensitive to changes in certain parameters. (get exact statement)

Step 3:
Is there a predictive function/model/law?
Traditional statistical technique -
MA?
ARMA?
ARIMA?

Iterative NN technique -
Test existence of a predictive function using sequential division in data (and using a large test set?)


Before we get into any further tradition statistical technique selection – check assumptions.

a) what kind of variables – ordinal, continuous etc are required and
b) what kind of distribution is required – normal? whatever..
c) how many independent and dependent variables are accounted for.

No comments: