GTSRB Dirty-label Backdoor Baseline Evaluation

Bullet Hole Trigger

Undefended

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0.000 | |0.01 |0.287|0.288|0.261|0.279 |0.012 | |0.05 |0.557|0.685|0.465|0.569 |0.090 | |0.1 |0.839|0.821|0.801|0.820 |0.016 | |0.2 |0.914|0.889|0.926|0.910 |0.015 | |0.3 |0.928|0.951|0.940|0.940 |0.009 |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.922|0.922|0.926|0.9233333333|0.002309401077| |0.05 |0.909|0.906|0.895|0.9033333333|0.007371114796| |0.1 |0.894|0.894|0.899|0.8956666667|0.002886751346| |0.2 |0.89 |0.895|0.89 |0.8916666667|0.002886751346| |0.3 |0.889|0.888|0.887|0.888 |0.001 |

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.983|0.975|0.922|0.960 |0.03315116891| |0.01 |0.972|0.968|0.974|0.971 |0.003055050463| |0.05 |0.971|0.963|0.949|0.961 |0.01113552873| |0.1 |0.961|0.958|0.979|0.966 |0.01135781669| |0.2 |0.965|0.958|0.949|0.957 |0.008020806277| |0.3 |0.963|0.963|0.946|0.957 |0.009814954576|

Random Filter

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0 | |0.01 |0.103|0.268|0.031|0.134 |0.1215030864 | |0.05 |0.314|0.724|0.667|0.568 |0.2220953249 | |0.1 |0.844|0.607|0.783|0.745 |0.1230623148 | |0.2 |0.892|0.9 |0.607|0.800 |0.166902167 | |0.3 |0.957|0.953|0.897|0.936 |0.03354598833|

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.904|0.918|0.863|0.895 |0.02858321186| |0.05 |0.847|0.899|0.906|0.884 |0.03223352292| |0.1 |0.893|0.888|0.9 |0.894 |0.006027713773| |0.2 |0.888|0.883|0.845|0.872 |0.02351595203| |0.3 |0.889|0.889|0.878|0.885 |0.006350852961|

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.964|0.982|0.964|0.970 |0.008 | |0.01 |0.944|0.971|0.925|0.947 |0.019 | |0.05 |0.879|0.968|0.974|0.940 |0.043 | |0.1 |0.972|0.94 |0.963|0.958 |0.013 | |0.2 |0.964|0.954|0.814|0.911 |0.068 | |0.3 |0.958|0.96 |0.936|0.951 |0.011 |

Activation Clustering

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0 | |0.01 |0.267|0.322|0.278|0.289 |0.02910326442| |0.05 |0.718|0.668|0.686|0.691 |0.02532455988| |0.1 |0.835|0.856|0.756|0.816 |0.05272886622| |0.2 |0.919|0.668|0.925|0.837 |0.1466776511 | |0.3 |0.919|0.918|0.653|0.830 |0.1532873119 |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.912|0.914|0.911|0.912 |0.001527525232| |0.05 |0.887|0.892|0.896|0.892 |0.004509249753| |0.1 |0.889|0.892|0.88 |0.887 |0.006244997998| |0.2 |0.873|0.846|0.88 |0.866 |0.01795364401| |0.3 |0.875|0.888|0.825|0.863 |0.03326158946|

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.974|0.976|0.868|0.939 |0.050 | |0.01 |0.929|0.978|0.969|0.959 |0.021 | |0.05 |0.965|0.94 |0.958|0.954 |0.011 | |0.1 |0.953|0.963|0.931|0.949 |0.013 | |0.2 |0.96 |0.858|0.958|0.925 |0.048 | |0.3 |0.969|0.956|0.749|0.891 |0.101 |

Perfect Filter

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0 | |0.01 |0.025|0.185|0.1 |0.103 |0.08005206639| |0.05 |0.146|0.026|0.111|0.094 |0.06171169527| |0.1 |0.138|0.082|0.057|0.092 |0.0414769012 | |0.2 |0.135|0.124|0.146|0.135 |0.011 | |0.3 |0.16 |0.165|0.081|0.135 |0.04712041313|

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.873|0.93 |0.934|0.912 |0.03412232896| |0.05 |0.926|0.876|0.934|0.912 |0.03143246729| |0.1 |0.938|0.917|0.918|0.924 |0.0118462371 | |0.2 |0.935|0.928|0.933|0.932 |0.003605551275| |0.3 |0.931|0.931|0.938|0.933 |0.004041451884|

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.979|0.974|0.982|0.978 |0.003 | |0.01 |0.919|0.981|0.975|0.958 |0.028 | |0.05 |0.976|0.892|0.972|0.947 |0.039 | |0.1 |0.976|0.956|0.961|0.964 |0.008 | |0.2 |0.978|0.972|0.981|0.977 |0.004 | |0.3 |0.969|0.974|0.979|0.974 |0.004 |

Spectral Signatures

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 | | | |0.000 | | |0.01 | | | |0.268 | | |0.05 | | | |0.562 | | |0.1 | | | |0.819 | | |0.2 | | | |0.823 | | |0.3 | | | |0.915 | |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 | | | |- | | |0.01 | | | |0.9261810504| | |0.05 | | | |0.9002639219| | |0.1 | | | |0.8919503827| | |0.2 | | | |0.8726576933| | |0.3 | | | |0.8911850092| |

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 | | | |0.970 | | |0.01 | | | |0.965 | | |0.05 | | | |0.960 | | |0.1 | | | |0.959 | | |0.2 | | | |0.920 | | |0.3 | | | |0.944 | |

Peace Sign Trigger

Undefended

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0.000 | |0.01 |0.590|0.772|0.868|0.743 |0.115 | |0.05 |0.950|0.978|0.932|0.953 |0.019 | |0.1 |0.981|0.986|0.982|0.983 |0.002 | |0.2 |0.930|0.992|0.979|0.967 |0.027 | |0.3 |0.999|0.996|0.999|0.998 |0.001 |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.901|0.897|0.89 |0.896 |0.005567764363| |0.05 |0.89 |0.891|0.893|0.8913333333|0.001527525232| |0.1 |0.886|0.885|0.888|0.8863333333|0.001527525232| |0.2 |0.861|0.887|0.87 |0.8726666667|0.01320353488| |0.3 |0.887|0.887|0.885|0.8863333333|0.001154700538|

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.975|0.942|0.965|0.961 |0.01692138686| |0.01 |0.972|0.981|0.969|0.974 |0.006244997998| |0.05 |0.979|0.974|0.983|0.979 |0.004509249753| |0.1 |0.978|0.969|0.982|0.976 |0.006658328118| |0.2 |0.924|0.969|0.951|0.948 |0.02264950331| |0.3 |0.967|0.971|0.971|0.970 |0.002309401077|

Random Filter

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0.000 | |0.01 |0.689|0.746|0.569|0.668 |0.074 | |0.05 |0.951|0.956|0.351|0.753 |0.284 | |0.1 |0.985|0.967|0.985|0.979 |0.008 | |0.2 |0.872|0.354|0.988|0.738 |0.276 | |0.3 |1 |0.997|0.996|0.998 |0.002 |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.903|0.897|0.886|0.895 |0.008621678104| |0.05 |0.889|0.89 |0.87 |0.883 |0.01126942767| |0.1 |0.884|0.888|0.89 |0.887 |0.003055050463| |0.2 |0.82 |0.858|0.889|0.856 |0.03455912808| |0.3 |0.844|0.885|0.885|0.871 |0.02367136104|

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.929|0.965|0.912|0.935 |0.02706165799| |0.01 |0.975|0.975|0.949|0.966 |0.015011107 | |0.05 |0.982|0.968|0.925|0.958 |0.02970409624| |0.1 |0.965|0.967|0.978|0.970 |0.007 | |0.2 |0.828|0.824|0.982|0.878 |0.09008884504| |0.3 |0.9 |0.968|0.975|0.948 |0.04142865353|

Activation Clustering

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0.000 | |0.01 |0.644|0.483|0.364|0.497 |0.115 | |0.05 |0.938|0.947|0.772|0.886 |0.080 | |0.1 |0.974|0.979|0.978|0.977 |0.002 | |0.2 |0.979|0.988|0.989|0.985 |0.004 | |0.3 |1 |0.996|0.969|0.988 |0.014 |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.893|0.866|0.781|0.847 |0.05844940832| |0.05 |0.876|0.882|0.864|0.874 |0.00916515139| |0.1 |0.88 |0.882|0.874|0.879 |0.004163331999| |0.2 |0.878|0.879|0.877|0.878 |0.001 | |0.3 |0.873|0.882|0.848|0.868 |0.01761628035|

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.971|0.969|0.964|0.968 |0.003 | |0.01 |0.975|0.94 |0.968|0.961 |0.015 | |0.05 |0.974|0.965|0.933|0.957 |0.018 | |0.1 |0.965|0.969|0.938|0.957 |0.014 | |0.2 |0.969|0.968|0.978|0.972 |0.004 | |0.3 |0.904|0.963|0.899|0.922 |0.029 |

Perfect Filter

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.000|0.000|0.000|0.000 |0.000 | |0.01 |0.032|0.221|0.158|0.137 |0.079 | |0.05 |0.076|0.019|0.039|0.045 |0.024 | |0.1 |0.024|0.157|0.036|0.072 |0.060 | |0.2 |0.35 |0.138|0.196|0.228 |0.089 | |0.3 |0.189|0.074|0.097|0.120 |0.050 |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |- |- |- |- |- | |0.01 |0.921|0.91 |0.911|0.914 |0.00608276253| |0.05 |0.848|0.881|0.925|0.885 |0.03863073043| |0.1 |0.906|0.854|0.922|0.894 |0.03555277767| |0.2 |0.914|0.917|0.909|0.913 |0.004041451884| |0.3 |0.91 |0.92 |0.911|0.914 |0.005507570547|

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 |0.979|0.986|0.981|0.982 |0.003 | |0.01 |0.981|0.982|0.974|0.979 |0.004 | |0.05 |0.881|0.907|0.972|0.920 |0.038 | |0.1 |0.979|0.904|0.967|0.950 |0.033 | |0.2 |0.968|0.981|0.971|0.973 |0.006 | |0.3 |0.971|0.972|0.958|0.967 |0.006 |

Spectral Signatures

Attack Success Rate |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 | | | |0.000 | | |0.01 | | | |0.270 | | |0.05 | | | |0.774 | | |0.1 | | | |0.756 | | |0.2 | | | |0.983 | | |0.3 | | | |0.992 | |

Accuracy on Poisoned Test Data All Classes |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 | | | |- | | |0.01 | | | |0.8585114806| | |0.05 | | | |0.8861177092| | |0.1 | | | |0.8614410135| | |0.2 | | | |0.8864080232| | |0.3 | | | |0.8678807073| |

Accuracy on Benign Test Data Source Class |Poison Ratio |Run 1|Run 2|Run 3|Mean |Std | |-------------------|-----|-----|-----|------------|-------------| |0 | | | |0.972 | | |0.01 | | | |0.921 | | |0.05 | | | |0.949 | | |0.1 | | | |0.913 | | |0.2 | | | |0.966 | | |0.3 | | | |0.944 | |