Results from simulation study

Pillar 1: Well-known systems

Binary classification measures from 100 realizations of system 1 for increasing time series length n. Overall results are extracted as a mean over all realizations and all time series lengths.

CGCI	Sensitivity	Specificity	F1-score	PMIME	Sensitivity	Specificity	F1-score
n=512	100	94.38	95.39	n=512	99.86	88.77	90.88
n=1024	100	94.38	95.33	n=1024	100	87	89.77
n=2048	100	95.54	96.28	n=2048	100	87.54	90.11
n=4096	100	94.92	95.77	n=4096	100	87.38	90.12
OVERALL	100	94.81	95.69	OVERALL	99.97	87.67	90.22
RCGCI	Sensitivity	Specificity	F1-score	PTENUE	Sensitivity	Specificity	F1-score
n=512	100	93.62	94.76	n=512	99.14	97.69	97.57
n=1024	100	95.77	96.42	n=1024	100	97.31	97.74
n=2048	100	96.15	96.77	n=2048	100	98	98.28
n=4096	100	97.15	97.6	n=4096	100	97.15	97.58
OVERALL	100	95.67	96.39	OVERALL	99.79	97.54	97.79

Both linear and nonlinear causality measures have high sensitivity and high specificity. PTENUE achieves the highest overall performance.

2. Nonlinear stochastic system

Binary classification measures from 100 realizations of system 2 for increasing time series length n. Overall results are extracted as a mean over all realizations and all time series lengths.

CGCI	Sensitivity	Specificity	F1-score	PMIME	Sensitivity	Specificity	F1-score
n=512	54.2	82.33	51.8	n=512	82.2	89.8	77.9
n=1024	62	80.2	55.69	n=1024	90.4	88.73	81.29
n=2048	64.8	78.8	56.69	n=2048	99.8	85.6	83.01
n=4096	65.8	76	55.23	n=4096	100	84.07	81.45
OVERALL	61.7	79.33	54.85	OVERALL	93.1	87.05	80.91
RCGCI	Sensitivity	Specificity	F1-score	PTENUE	Sensitivity	Specificity	F1-score
n=512	54.4	84.73	53.52	n=512	81.4	93.6	81.75
n=1024	59.6	82.87	55.93	n=1024	86.8	90.6	81.08
n=2048	67.4	82.2	60.67	n=2048	99.4	85.2	81.92
n=4096	66	79.8	57.96	n=4096	100	85.27	82.25
OVERALL	61.85	82.4	57.02	OVERALL	91.9	88.67	81.75

Causality measures havean increasing sensitivity over sample size, while present decreasing specificity due to inference of spurious and indirect causal effects. PMIME has the optimal sensitivity, PTENUE the optimal specificity. Overall, PTENUE achieves the highestperformance.

3. Chaotic system

Coupled Henon maps, K=5 variables

Binary classification measures from 100 realizations of system 3 in K=5 variables for increasing time series lengthn and coupling strength c.

CGCI	n=512				RCGCI		n=512
Coupling strength	Sensitivity	Specificity	F1-score	Coupling strength		Sensitivity		Specificity	F1-score
c = 0	–	88.4	–	c = 0		–		94.25	–
c = 0.1	49.33	91.29	55.22	c = 0.1		33.33		95.5	36.34
c = 0.2	85	85.07	77.5	c = 0.2		78.17		91.93	71.54
c = 0.3	81.33	84.21	75.02	c = 0.3		62.33		91	57.66
c = 0.4	99	65.86	71.71	c = 0.4		95.33		82	73.06
c = 0.5	100	70.21	74.65	c = 0.5		99.5		81.21	75.18
Coupling strength	n=1024				Coupling strength		n=1024
c = 0	–	90.85	–	c = 0		–		96.8	–
c = 0.1	74.5	90.5	74.94	c = 0.1		55.83		96.93	61.08
c = 0.2	94.83	78.57	78.27	c = 0.2		88.17		92.14	79.67
c = 0.3	89.33	72.29	70.7	c = 0.3		78.17		87.93	65.98
c = 0.4	100	53.86	65.28	c = 0.4		99.83		71.14	65.5
c = 0.5	100	63	70.22	c = 0.5		100		77.14	71.56
Coupling strength	n=2048				Coupling strength		n=2048
c = 0	–	88.3	–	c = 0		–		96.65	–
c = 0.1	94.33	89	86.12	c = 0.1		83.83		97.14	83.88
c = 0.2	99.67	70.14	74.61	c = 0.2		96.17		90.93	84.27
c = 0.3	97.67	58.64	66.77	c = 0.3		86.67		84.71	68.56
c = 0.4	100	48.14	62.47	c = 0.4		100		60	56.05
c = 0.5	100	53.21	65.05	c = 0.5		100		63.64	59.33
Coupling strength	n=4096				Coupling strength		n=4096
c = 0	–	88.25	–	c = 0		–		98	–
c = 0.1	99.33	89.63	87.09	c = 0.1		98		97.14	94.32
c = 0.2	99.83	59.14	67.96	c = 0.2		99		88.21	83.04
c = 0.3	100	45.43	61.27	c = 0.3		92.83		76.14	64.24
c = 0.4	100	41.64	59.69	c = 0.4		100		51.21	49.2
c = 0.5	100	39	58.67	c = 0.5		100		47.43	46.34
PMIME	n=512				PTENUE		n=512
Coupling strength	Sensitivity	Specificity	F1-score	Coupling strength		Sensitivity		Specificity	F1-score
c = 0	–	99.6	–	c = 0		–		100	–
c = 0.1	49.67	99.71	62.67	c = 0.1		31.17		99.93	46.11
c = 0.2	98	99.64	98.03	c = 0.2		97.17		99.93	97.88
c = 0.3	100	99.64	99.46	c = 0.3		100		99.93	99.89
c = 0.4	100	99.86	99.78	c = 0.4		100		100	100
c = 0.5	100	98.07	97.16	c = 0.5		100		99.5	99.26
Coupling strength	n=1024				Coupling strength		n=1024
c = 0	–	99.9	–	c = 0		–		100	–
c = 0.1	85.33	99.86	89.38	c = 0.1		71.67		100	79.84
c = 0.2	100	99.86	99.78	c = 0.2		99.83		99.93	99.76
c = 0.3	100	100	100	c = 0.3		100		100	100
c = 0.4	100	99.93	99.89	c = 0.4		100		100	100
c = 0.5	100	98.71	98.08	c = 0.5		100		99.43	99.14
Coupling strength	n=2048				Coupling strength		n=2048
c = 0	–	100	–	c = 0		–		100	–
c = 0.1	100	100	100	c = 0.1		99.67		100	99.76
c = 0.2	100	99.93	99.89	c = 0.2		100		100	100
c = 0.3	100	100	100	c = 0.3		100		100	100
c = 0.4	100	100	100	c = 0.4		100		100	100
c = 0.5	100	99.5	99.28	c = 0.5		100		99.86	998
Coupling strength	n=4096				Coupling strength		n=4096
c = 0	–	100	–	c = 0		–		100	–
c = 0.1	100	99.93	99.89	c = 0.1		100		99.93	99.89
c = 0.2	100	100	100	c = 0.2		100		100	100
c = 0.3	100	100	100	c = 0.3		100		100	100
c = 0.4	100	99.5	99.28	c = 0.4		100		99.5	99.28
c = 0.5	100	100	100	c = 0.5		100		100	100

Causality measures havean increasing sensitivity over sample size, while present decreasing specificity. For low coupling strengths, causality measures require larger sample size to correctly infer causality. PMIME and PTENUE achieve the optimal performance.

Coupled Henon maps, K=15 variables

Binary classification measures from 100 realizations of system 3 in K=15 variables for increasing time series length n and coupling strength c.

CGCI	n=512				RCGCI		n=512
Coupling strength	Sensitivity	Specificity	F1-score	Coupling strength		Sensitivity		Specificity	F1-score
c = 0	–	88.4	–	c = 0		–		94.25	–
c = 0.1	49.33	91.29	45.64	c = 0.1		33.33		95.5	36.34
c = 0.2	85	85.07	68.39	c = 0.2		78.17		91.93	71.54
c = 0.3	81.33	84.21	64.34	c = 0.3		62.33		91	57.66
c = 0.4	99	65.86	60.38	c = 0.4		95.33		82	73.06
c = 0.5	100	70.21	64.83	c = 0.5		99.5		81.21	75.18
Coupling strength	n=1024				Coupling strength		n=1024
c = 0	–	90.85	–	c = 0		–		96.8	–
c = 0.1	74.5	90.5	66.53	c = 0.1		55.83		96.93	61.08
c = 0.2	94.83	78.57	69.01	c = 0.2		88.17		92.14	79.67
c = 0.3	89.33	72.29	57.36	c = 0.3		78.17		87.93	65.98
c = 0.4	100	53.86	51.14	c = 0.4		99.83		71.14	65.5
c = 0.5	100	63	58.49	c = 0.5		100		77.14	71.56
Coupling strength	n=2048				Coupling strength		n=2048
c = 0	–	88.3	–	c = 0		–		96.65	–
c = 0.1	94.33	89	80.7	c = 0.1		83.83		97.14	83.88
c = 0.2	99.67	70.14	64.68	c = 0.2		96.17		90.93	84.27
c = 0.3	97.67	58.64	52.74	c = 0.3		86.67		84.71	68.56
c = 0.4	100	48.14	46.77	c = 0.4		100		60	56.05
c = 0.5	100	53.21	50.71	c = 0.5		100		63.64	59.33
Coupling strength	n=4096				Coupling strength		n=4096
c = 0	–	88.25	–	c = 0		–		98	–
c = 0.1	99.33	89.63	82.01	c = 0.1		98		97.14	94.32
c = 0.2	99.83	59.14	55.15	c = 0.2		99		88.21	83.04
c = 0.3	100	45.43	44.78	c = 0.3		92.83		76.14	64.24
c = 0.4	100	41.64	42.05	c = 0.4		100		51.21	49.2
c = 0.5	100	39	40.17	c = 0.5		100		47.43	46.34
PMIME	n=512				PTENUE		n=512
Coupling strength	Sensitivity	Specificity	F1-score	Coupling strength		Sensitivity		Specificity	F1-score
c = 0	–	99.6	–	c = 0		–		100	–
c = 0.1	49.67	99.71	62.67	c = 0.1		31.17		99.93	46.11
c = 0.2	98	99.64	98.03	c = 0.2		97.17		99.93	97.88
c = 0.3	100	99.64	99.46	c = 0.3		100		99.93	99.89
c = 0.4	100	99.86	99.78	c = 0.4		100		100	100
c = 0.5	100	98.07	97.16	c = 0.5		100		99.5	99.26
Coupling strength	n=1024				Coupling strength		n=1024
c = 0	–	99.9	–	c = 0		–		100	–
c = 0.1	85.33	99.86	89.38	c = 0.1		71.67		100	79.84
c = 0.2	100	99.86	99.78	c = 0.2		99.83		99.93	99.76
c = 0.3	100	100	100	c = 0.3		100		100	100
c = 0.4	100	99.93	99.89	c = 0.4		100		100	100
c = 0.5	100	98.71	98.08	c = 0.5		100		99.43	99.14
Coupling strength	n=2048				Coupling strength		n=2048
c = 0	–	100	–	c = 0		–		100	–
c = 0.1	100	100	100	c = 0.1		99.67		100	99.76
c = 0.2	100	99.93	99.89	c = 0.2		100		100	100
c = 0.3	100	100	100	c = 0.3		100		100	100
c = 0.4	100	100	100	c = 0.4		100		100	100
c = 0.5	100	99.5	99.28	c = 0.5		100		99.86	99.8
Coupling strength	n=4096				Coupling strength		n=4096
c = 0	–	100	–	c = 0		–		100	–
c = 0.1	100	100	100	c = 0.1		100		100	100
c = 0.2	100	100	100	c = 0.2		100		100	100
c = 0.3	100	100	100	c = 0.3		100		100	100
c = 0.4	100	100	100	c = 0.4		100		100	100
c = 0.5	100	100	100	c = 0.5		100		100	100

Linear causality measures havean increasing sensitivity over sample size, while present decreasing specificity. PMIME and PTENUE achieve the optimal performance.

Pillar 2: Systems of known network types

1. Random networks

VAR(1) in 15 variables

The exported results of network measures based on the true connectivity network and the extracted connectivity networks based on the causality measures RCGCI, PMIME and PTENUE for time series length of 500, 1000, 2000 and 4500.

Networks created using PMIME and PTENUE (a = 0.05) significantly over- or under-estimate network connectivity with the closest value to true connectivity being 12% close and the average difference between them being ~48%. On the other hand, PTENUEa001 results in quite similar results, while RCGCI almost replicates true connectivity for all examined time series lengths.

VAR(5) in 15 variables

No causality measure manages to completely replicate the behavior of true connectivity, especially the nonlinear measures PMIME and PTENUE (a = 0.05). The best results are given by PTENUE (a = 0.01), whose results for the shortest sample size (500) are on average as close as 7% to true connectivity (if we exclude assortativity from the average value). For sample size 1000, the average distance from true connectivity becomes 11%. Similar value (12%) is also acquired for RCGCI, for the largest sample size (4500).

Coupled Henon maps, 15 variables

Networks created using PMIME and PTENUE (significance level a =0.05) are almost the same to the initial, true connectivity networks, while PTENUE (significance level a =0.05) results are also quite acceptable. The best results are acquired when sample sizes are the longest one examined (i.e. 4500). On the other hand, RCGCI results are significantly different for all sample sizes.

2. Small-world networks

VAR(1) in 15 variables

Networks created using PMIME and PTENUE (a = 0.05) significantly over- or under-estimate network connectivity with the closest value to true connectivity being 25% close and the average difference between them being ~80%. PTENUE (a = 0.01) results are quite better, and it outstands in performance for integration measures. Finally, the best results are acquired by RCGCI, which almost replicates true connectivity, especially for large sample sizes (2000 and 45000), with average differences compared to true connectivity of 4% and 3%, respectively.

VAR(5) in 15 variables

RCGCI for sample size 4500 and PTENUE (a = 0.01) for sample size 500 are the ones that outstand, however, with significant changes compared to true connectivity. To be more specific, the first one differs on average by 18% from true connectivity, with significantly lower differences (better results) for integration measures, while the latter one differs by 15%. The other two measures completely fail to replicate the true connectivity behavior, with all the values being almost twice or half, compared to the expected ones.

Coupled Henon maps, 15 variables

Networks created using PMIME, PTENUE (a = 0.05) and PTENUE (a = 0.01) are almost identical to the initial, true connectivity networks, especially for sample sizes greater than 2000, although for smaller ones the results are also acceptable. On the other hand, RCGCI results are significantly different for all sample sizes.

3. Scale-free networks

VAR(1) in 15 variables

Networks created using PMIME and PTENUE (a = 0.05) significantly over- or under-estimate network connectivity with the closest value to true connectivity being 10% close and the average difference between them being ~30%. On the other hand, PTENUE (a = 0.01) results in quite similar results for sample size 2000 with true connectivity. Finally, the best results are acquired by RCGCI, which almost replicates true connectivity for all examined time series lengths. Even more, for sample size of 4500 the average diversion from true connectivity is less than 1%.

VAR(5) in 15 variables

he exported results of network measures based on the true connectivity network and the extracted connectivity networks based on the causality measures RCGCI, PMIME and PTENUE for time series length of 500, 1000, 2000 and 4500.

PMIME and PTENUE (a = 0.05) again fail to replicate the behavior of true connectivity, since their results differ on average by 50%. RCGCI for sample size of 4500 gives quite respectable results with the discrepancies from true connectivity being on average 8%. For integration measures, the differences are even smaller. Finally, PTENUE (a = 0.01) for sample size 1000 gives the best results that replicate the true connectivity behavior without over- or under-estimating significantly the measures.

Coupled Henon maps, 15 variables

Networks created using PMIME and PTENUE (a = 0.05) are almost the same to the initial, true connectivity networks, while PTENUEa001 results are also quite acceptable. The best results are acquired when sample sizes are the longest one examined (i.e. 4500). On the other hand, RCGCI results are significantly different for all sample sizes.

Pillar 3: Scale-free networks of Pillar 2 augmented with random variables

1. Scale free networks augmented with random and uncorrelated variables

VAR(1) in 20 variables

PMIME and PTENUE significantly fail to replicate the behavior of true connectivity, except for the integration measures, transitivity and modularity where they get quite good approximations. One the other hand, RCGCI almost completely replicates true connectivity, for all the measures, especially for the lengthiest timeseries examined, whose values are almost identical to the true ones, with only minor discrepancies.

VAR(5) in 20 variables

All three causality measures fail to replicate the basic characteristics of the real network (e.g. number of nodes and edges, density), especially PMIME and PTENUE. PTENUE, however, gest very good results for average betweenness centrality, while RCGCI gets better values for closeness centrality. RCGCI also outstands in performance for segregation measures and average local efficiency, especially for timeseries of length 4500.

iii. Coupled Henon maps, 20 variables

There is no causality measure that outstands in performance, but PMIME is the one that differs on average the least by true connectivity.

2. Scale free networks augmented with 5 random and highly correlated variables

VAR(1) in 20 variables

All three causality measures fail to replicate the basic characteristics of the real network (e.g. number of nodes and edges), especially PMIME and PTENUE. These two also fail to replicate the centrality measures, while RCGCI gives significantly better results. On the other hand, all three measures give quite good results for integration and segregation measures, in which PTENUE for timeseries of length 4500 outstands in performance. Altogether, RCGCI gives the best results, that differ less by true connectivity, especially for longer timeseries.

VAR(5) in 20 variables

All three causality measures fail to replicate the basic characteristics of the real network (e.g. number of nodes and edges), especially PMIME and PTENUE. These two also fail to replicate the centrality measures, while RCGCI gives significantly better results for closeness centrality. On the other hand, all three measures give quite good results for integration and segregation measures, in which RCGCI for timeseries of length 4500 outstands in performance. Altogether, RCGCI gives the best results, however, they show quite significant discrepancies from true connectivity.

iii. Coupled Henon maps, 20 variables

Networks created using PMIME and PTENUE are the ones that are closer to true connectivity networks, mainly for short timeseries. However, there are measures such as average betweenness centrality and modularity where RCGCI values are better, compared to PMIME and PTENUE.