Tuesday, November 12, 2013

FDR-based undirected edge inference validation

Right now, I suspect that the G-causality based inference is "too eager" to predict an edge for a given FDR level $q$, while the undirected inference results seems to be better calibrated to $q$. Here, I'd like to explicitly quantify the FDR calibration for undirected edge inference.

Here is a single instance of $N=100$ undirected inference, where I shmoo the FDR level q, and measure the actual FDR observed by the inference method. I also plot the standard ROC of the inference:

Note that the FDR is not the same quantity as the FPR. See the Wikipedia page on ROC for details.

Here is the above graph, based on 50 inference runs:


It is interesting that the undirected inference for the choice of parameters that I've picked for the Izhikevich numerical model performs so well. This way, we can see the effects of temporal filtering on the inference performance!

No comments:

Post a Comment