Mutagenesis Example
Following example demonstrates learning to predict the mutagenicity on Salmonella typhimurium (dataset is stored in json format in MLDatasets.jl for your convenience).
This example is also available as a Jupyter notebook, feel free to run it yourself: mutagenesis.ipynb
This example is taken from the CTUAvastLab/JsonGrinderExamples and heavily commented for more clarity.
Here we include libraries all necessary libraries
using JsonGrinder, Mill, Flux, MLDatasets, Statistics, Random, JSON3, OneHotArrayswe stabilize the seed to obtain same results every run, for pedagogic purposes
Random.seed!(42)Random.TaskLocalRNG()We define the minibatch size.
BATCH_SIZE = 1010Here we load the training samples.
dataset_train = MLDatasets.Mutagenesis(split=:train);
x_train = dataset_train.features
y_train = dataset_train.targets100-element Vector{Int64}:
1
1
0
1
1
1
1
1
1
1
1
1
1
0
0
1
0
0
1
0
1
1
0
1
0
1
1
0
0
1
1
1
1
0
1
0
0
0
0
1
0
1
1
0
0
0
1
0
0
0
0
1
1
0
1
1
1
0
1
1
1
0
0
0
1
1
1
1
0
1
1
1
1
1
1
1
0
1
0
1
1
1
1
1
0
1
1
1
0
1
1
0
0
1
1
0
0
1
0
0This is the step 1 of the workflow
We create the schema of the training data, which is the first important step in using the JsonGrinder. This computes both the structure (also known as JSON schema) and histogram of occurrences of individual values in the training data.
sch = JsonGrinder.schema(x_train)[Dict] # updated = 100
├─── lumo: [Scalar - Float64], 98 unique values # updated = 100
├─── inda: [Scalar - Int64], 1 unique values # updated = 100
├─── logp: [Scalar - Float64,Int64], 62 unique values # updated = 100
├─── ind1: [Scalar - Int64], 2 unique values # updated = 100
╰── atoms: [List] # updated = 100
╰── [Dict] # updated = 2529
├──── element: [Scalar - String], 6 unique values # updated = 2529
├────── bonds: [List] # updated = 2529
│ ╰── [Dict] # updated = 5402
│ ┊
├───── charge: [Scalar - Float64], 318 unique values # updated = 2529
╰── atom_type: [Scalar - Int64], 28 unique values # updated = 2529This is the step 2 of the workflow
Then we use it to create the extractor converting jsons to Mill structures. The suggestextractor is executed below with default setting, but it allows you heavy customization.
extractor = suggestextractor(sch)Dict
├─── lumo: Categorical d = 99
├─── inda: Categorical d = 2
├─── logp: Categorical d = 63
├─── ind1: Categorical d = 3
╰── atoms: Array of
╰── Dict
├──── element: Categorical d = 7
├────── bonds: Array of
│ ╰── Dict
│ ┊
├───── charge: Float32
╰── atom_type: Categorical d = 29This is the step 3 of the workflow, we create the model using the schema and extractor
Create the model
We create the model reflecting structure of the data
encoder = reflectinmodel(sch, extractor, d -> Dense(d, 10, relu))ProductModel ↦ Dense(50 => 10, relu) # 2 arrays, 510 params, 2.070 KiB
├─── lumo: ArrayModel(Dense(99 => 10, relu)) # 2 arrays, 1_000 params, 3.984 KiB
├─── inda: ArrayModel(Dense(2 => 10, relu)) # 2 arrays, 30 params, 200 bytes
├─── logp: ArrayModel(Dense(63 => 10, relu)) # 2 arrays, 640 params, 2.578 KiB
├─── ind1: ArrayModel(Dense(3 => 10, relu)) # 2 arrays, 40 params, 240 bytes
╰── atoms: BagModel ↦ BagCount([SegmentedMean(10); SegmentedMax(10)]) ↦ Dense(21 => 10, relu) # 4 arrays, 240 params, 1.094 KiB
╰── ProductModel ↦ Dense(31 => 10, relu) # 2 arrays, 320 params, 1.328 KiB
├──── element: ArrayModel(Dense(7 => 10, relu)) # 2 arrays, 80 params, 400 bytes
├────── bonds: BagModel ↦ BagCount([SegmentedMean(10); SegmentedMax(10)]) ↦ Dense(21 => 10, relu) # 4 arrays, 240 params, 1.094 KiB
│ ╰── ProductModel ↦ Dense(31 => 10, relu) # 2 arrays, 320 params, 1.328 KiB
│ ┊
├───── charge: ArrayModel(identity)
╰── atom_type: ArrayModel(Dense(29 => 10, relu)) # 2 arrays, 300 params, 1.250 KiBthis allows us to create model flexibly, without the need to hardcode individual layers. Individual arguments of reflectinmodel are explained in Mill.jl documentation. But briefly: for every numeric array in the sample, model will create a dense layer with neurons neurons (20 in this example). For every vector of observations (called bag in Multiple Instance Learning terminology), it will create aggregation function which will take mean, maximum of feature vectors and concatenate them. The fsm keyword argument basically says that on the end of the NN, as a last layer, we want 2 neurons length(labelnames) in the output layer, not 20 as in the intermediate layers. then we add layer with 2 output of the model at the end of the neural network
model = Dense(10, 2) ∘ encoderDense(10 => 2) ∘ ProductModel ↦ Dense(50 => 10, relu)This is the step 4 of the workflow, we call the extractor on each sample
We convert jsons to mill data samples and prepare list of classes. This classification problem is two-class, but we want to infer it from labels. The extractor is callable, so we can pass it vector of samples to obtain vector of structures with extracted features.
ds_train = extractor.(x_train)100-element Vector{ProductNode{NamedTuple{(:lumo, :inda, :logp, :ind1, :atoms), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bonds, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bond_type, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}}}, Nothing}}:
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNodeThis is the step 5 of the workflow, we train the model
Train the model
Then, we define few handy functions and a loss function, which is logit binary crossentropy in our case. Here we add +1 to labels, because the labels are {0,1} and idxmax of the model output is in the {1,2} range.
loss(ds, y) = Flux.Losses.logitbinarycrossentropy(model(ds), OneHotArrays.onehotbatch(y .+ 1, 1:2))
accuracy(ds, y) = mean(Flux.onecold(model(ds)) .== y .+ 1)accuracy (generic function with 1 method)We prepare the optimizer.
opt = AdaBelief()
ps = Flux.params(model)Params([Float32[-0.14937238 0.30904353 -0.41546988 0.3405928 0.6001251 0.5224303 -0.092750885 0.45076948 -0.20876926 0.39260277; -0.6390362 -0.10578741 0.124983445 -0.59301573 0.69497174 -0.7057435 0.004279422 -0.23132053 -0.22353533 -0.038621854], Float32[0.0, 0.0], Float32[-0.096426584 0.07578926 -0.06996167 -0.09683977 -0.18165171 -0.21285167 -0.18779413 -0.16627313 -0.18487893 -0.16527334 -0.11375091 0.21801732 -0.12990144 -0.112366766 -0.23416406 0.22334129 0.1458529 0.19612898 -0.09759484 -0.15168947 -0.16360721 -0.027514614 -0.039260298 0.20932846 -0.22651687 0.22665685 0.035197116 -0.17270982 -0.15612908 -0.22658098 -0.02221678 0.18217307 -0.1564237 -0.11346426 0.032175235 -0.17583966 0.11603635 -0.0030727547 0.11360248 0.17170793 0.08548008 0.11031378 0.065614745 -0.13627854 -0.20864525 -0.10172736 0.082583584 -0.21782163 0.12655717 0.0138090765 0.03547233 -0.08930749 0.10285816 -0.19322488 -0.012325477 -0.13909414 -0.106226794 -0.2163731 0.054409776 -0.16964208 -0.16506134 0.05264383 -0.21107616 0.0529773 -0.04251331 -0.18875185 -0.19109201 -0.101429656 0.22793299 0.024720343 0.040361594 -0.20213515 0.14131917 0.19981828 0.14099272 0.130157 -0.05187441 0.023193 0.1616941 -0.08250345 -0.111259736 0.040550355 -0.21764277 0.03890087 -0.02437135 -0.2019517 0.09136347 0.13882263 0.05816986 -0.10446782 0.20230137 0.0713801 0.11439332 0.09066509 0.11606496 -0.23421893 0.21728946 0.01820106 -0.037255753; -0.22306022 -0.2256287 0.1062214 0.027228577 -0.041388158 0.114647396 0.06919194 0.050866026 -0.041612383 -0.026956134 0.10877363 0.09125148 -0.036334883 0.15662871 0.121312924 -0.23312163 -0.043826554 0.23258606 0.1277648 -0.1533868 0.0534481 0.20202632 0.19420247 0.057553545 -0.15457419 0.089874774 0.13157682 0.16342606 -0.0887624 -0.10784051 -0.09284606 0.045813955 -0.14214355 0.054875538 -0.010774751 0.14675139 0.064102896 -0.15840603 -0.04745625 0.028785316 -0.09101979 -0.07820855 0.19675691 -0.10171816 0.10500865 -0.075222805 -0.05984175 0.033138283 -0.18911123 -0.08555322 0.12829392 -0.08801069 0.11460415 0.12826598 0.05974162 0.064277194 -0.16104302 0.09324308 -0.22886446 -0.15465826 0.16408378 0.2094592 -0.0011568699 0.007632634 -0.13190548 -0.17918576 0.16805784 -0.19866624 0.11173247 0.1371009 -0.0058231982 -0.22235927 0.05589156 -0.10383021 -0.15299761 0.21781486 -0.01470044 0.043741893 0.08132675 0.124703765 -0.029354926 -0.08596872 -0.028917328 -0.07845585 0.027632669 0.05748502 -0.002175742 0.14704543 -0.14754482 0.02064793 -0.16803922 0.0034726793 -0.19662322 0.16003226 0.22275832 -0.2000003 0.13838883 0.032021772 -0.103447095; 0.14084035 -0.15627764 -0.03206907 0.14527056 -0.20607188 -0.21399716 -0.19247118 0.14314944 0.108806856 0.09136817 -0.092668094 -0.19743906 0.115276605 0.09233191 -0.016215757 0.17352673 0.20168991 0.1100575 -0.064422466 0.118994154 -0.13117863 0.013815762 -0.055786926 0.020837922 0.1369123 -0.12245542 -0.05324443 0.023170372 -0.22117475 -0.09103355 -0.21830675 0.19399236 0.09012339 -0.16445369 0.07174327 -0.10157238 0.13695459 -0.16935025 0.06573526 0.22557327 -0.10754121 0.01614066 -0.09065622 -0.22848853 0.2314067 -0.2231013 -0.03870042 0.09106353 -0.06974242 -0.07639766 0.08417898 0.036457498 -0.14673519 0.10474275 0.14101236 -0.12201695 -0.12937771 -0.22108226 -0.21828146 -0.063776776 -0.046298765 -0.091449246 -0.098083004 -0.14958079 -0.15192099 0.036900073 -0.0060454933 -0.008874025 -0.14217566 0.062065456 0.11980978 -0.035791002 0.12894467 0.14897065 -0.12606137 -0.21980391 0.19322947 -0.110509925 0.08782193 -0.034668732 0.054194447 0.05171015 0.033847373 0.1873491 0.15852843 -0.19536678 -0.18093948 -0.09292418 -0.12030999 -0.19003242 0.13714282 0.08367459 -0.121257044 -0.0030244528 -0.21305372 -0.07766747 0.05534231 -0.13980508 0.10496449; 0.16756152 0.017439555 0.22197641 -0.18188474 0.014141457 -0.069731176 0.14450498 -0.1704676 0.18097402 0.010327447 -0.1598846 -0.13129418 -0.1503061 0.053810854 -0.17797746 0.06611866 0.09728626 -0.23457393 0.17519315 0.113242276 -0.1231774 0.07568479 -0.14736615 0.058907818 -0.11776873 -0.035351112 -0.13073812 0.14746423 -0.13964015 -0.20171945 -0.06343254 -0.08648816 -0.22289325 -0.17102675 0.013398357 -0.20314407 0.017282596 -0.1559859 -0.033998825 -0.12108744 0.21495797 0.09999517 0.17993583 -0.19040354 0.118755855 0.09877137 -0.17171803 0.07752846 0.23186192 0.15812632 0.22833392 0.2301421 -0.18006037 -0.09088199 0.06528301 0.21891811 -0.09874995 -0.037270967 0.09899263 -0.11478156 -0.21686646 -0.20699672 0.09249816 -0.12416425 0.07448703 -0.072465256 -0.033629637 0.21095906 0.06327854 0.17813794 0.007428798 0.15447372 0.13632128 0.15077761 -0.22870314 0.08676597 -0.09558792 -0.010076093 -0.043721225 0.05014245 0.13259651 0.22381216 0.052372646 0.14844358 -0.07507264 0.20923568 -0.20598398 -0.038061477 0.1887795 0.22080933 0.019046443 -0.13943335 -0.18239993 -0.22157472 -0.1542858 0.099143915 -0.20268792 -0.0065543843 -0.08440228; 0.048042476 -0.008182219 0.031957023 0.10901877 0.15211229 -0.16477431 0.2147255 0.21112415 0.22102821 0.0894899 -0.06197495 0.093648955 -0.038598336 -0.10424158 -0.13848536 -0.038362782 0.0767961 0.02590356 -0.12078689 0.1590764 -0.20676915 0.116574965 0.019915402 -0.10171486 -0.110356435 0.05931893 0.22475572 -0.19500375 -0.17814901 -0.10063739 0.11770336 -0.024404213 -0.06979643 -0.08687743 -0.03006371 -0.2158246 0.15376315 -0.23212822 0.06690466 -0.0024197132 -0.13000616 0.06910619 -0.18121973 -0.17963336 0.12286933 -0.038595874 -0.12350134 0.23105243 -0.0061150794 -0.122426726 -0.055964977 -0.06448522 0.12830158 0.051024836 0.06159603 0.22367766 -0.02532265 -0.025688201 -0.19689977 0.19040924 -0.1687808 0.16387036 -0.12987572 -0.15925108 0.022287149 0.14791556 -0.086462624 -0.051946796 0.05203073 -0.0059856405 0.18481055 -0.06974049 -0.16088964 -0.111467324 -0.18403648 0.09457092 -0.0032968961 -0.13551034 0.07022225 0.18721272 0.20621276 0.1107749 -0.024540449 0.17249148 0.038311515 0.19588062 -0.05608678 0.10551919 0.037851065 0.122617975 0.15659322 0.20106381 -0.18641606 0.18959431 -0.13710888 0.14790493 0.1420303 -0.0051251547 -0.19617312; 0.06287652 -0.091589764 -0.20875742 0.119152345 0.08218492 0.085991345 0.013900283 -0.17324291 -0.11553473 0.020469937 -0.123820126 0.08246508 0.2296697 -0.028186142 -0.09463849 0.056287985 -0.17099983 -0.2132552 0.19881918 -0.036806073 -0.2097018 0.05804364 -0.18145455 0.05100484 0.12503864 -0.027423771 0.17033084 0.032685693 0.17350551 -0.05927664 0.055994175 -0.1053876 0.18780035 -0.12201936 0.15640949 0.13683754 0.23196812 0.2018835 -0.026138581 -0.004622921 -0.12099369 0.21892871 -0.21181853 -0.14488068 0.20230475 0.1960437 -0.008138616 -0.06859235 0.22961096 -0.1803993 -0.052675296 0.15749711 -0.22254986 -0.20150097 -0.06544257 -0.13079989 0.034714933 0.059647255 0.16033289 0.07805741 0.10155731 0.20323153 -0.003165499 0.06435005 -0.17410652 -0.0751257 0.208176 0.14689171 -0.16981953 0.1695395 0.10802563 -0.06401586 -0.10444284 -0.14937246 -0.093442716 -0.16616993 -0.097568944 -0.22747774 0.10334708 -0.050102286 0.11184666 -0.21624875 -0.10644993 0.06776861 0.097526036 0.104886174 -0.1904204 -0.19487101 -0.02296869 0.026090112 0.16730474 -0.21505018 0.15530744 0.018647637 -0.13352403 -0.15301898 0.0005234345 -0.16558854 0.022711182; 0.018186852 -0.09998818 0.049838066 -0.06979335 0.018063035 -0.20910643 -0.13050406 -0.11784232 0.18166547 -0.12959653 -0.12468783 0.1634465 -0.020411903 -0.010607834 0.11902159 0.14585598 -0.22027275 -0.18647149 0.17888339 0.07355075 0.1728698 -0.11158144 -0.23077615 -0.13964653 -0.10580623 0.04498513 0.19867301 -0.14812617 0.11329136 -0.20215093 -0.18386964 0.19010796 0.2035005 -0.11150989 0.033492897 0.062279947 -0.1597437 -0.05698203 0.1874308 -0.083952904 0.028973015 0.13019146 0.081338726 -0.22492683 -0.0028932516 -0.057732124 0.036615495 0.15063818 0.06867127 0.050550066 0.10202005 0.011051138 0.069320455 0.08151787 -0.12329255 -0.028399935 -0.08443041 -0.15836942 0.123146355 0.2116907 0.04222635 -0.23325068 -0.047259714 -0.19144051 0.11559536 -0.13225879 0.1539774 -0.1721837 -0.20514669 0.082762554 -0.074212104 0.19776079 0.13212599 -0.11443738 -0.065302275 0.17254363 -0.040868666 0.2330871 0.19046398 0.15018117 0.093955636 0.2126063 0.041318685 0.10189651 -0.10086173 0.05979146 -0.06533799 -0.034146555 -0.22563833 0.1513796 -0.11699106 -0.09873255 -0.02650816 -0.002650455 0.11079456 0.16929153 0.17710651 -0.14541146 0.19325817; -0.0060510593 -0.02811622 -0.07473688 -0.1951872 -0.024415651 -0.18838008 -0.031265974 0.18147928 0.14128944 -0.019957636 0.10326186 0.15117013 -0.1535109 -0.12852147 0.15401468 0.00908393 -0.12743919 0.06256708 0.023665868 0.061524596 -0.07335981 0.21892545 -0.18140183 0.024667231 -0.104644634 -0.046874724 -0.16164549 -0.064083315 -0.012662499 0.031680804 0.030010011 -0.11621792 0.21129088 0.15582947 -0.06104622 0.08204278 -0.090648614 0.0320689 0.13875757 0.1970628 -0.08322247 -0.08364883 -0.08078226 -0.003279947 0.20049787 -0.038091067 0.23390429 0.091263115 -0.054305732 0.17768787 0.105373725 0.07544832 -0.009899974 0.22188103 0.060599335 0.14951487 -0.2046911 0.22522002 -0.11938691 -0.019126544 0.0025206802 -0.2149898 -0.1504207 0.19926596 0.22343414 -0.20948896 -0.20656747 -0.011086099 -0.11886413 0.15842414 -0.13463786 -0.18622778 -0.15373501 -0.096191674 0.119256414 -0.22117642 -0.013849967 -0.11876204 0.2197821 -0.02460363 0.19622497 -0.013054789 -0.12043224 0.07382459 0.20364752 0.07598979 -0.17354763 0.004281255 -0.030207554 -0.22246273 -0.0662677 -0.06901479 0.061339445 0.13984832 0.004679725 0.20221853 0.041941687 0.12074057 0.056234345; -0.16669913 -0.01743069 -0.14625864 -0.16003066 0.23303013 -0.00586269 0.18765122 -0.06958496 -0.17096578 -0.19252336 0.14970352 -0.038734656 0.010202232 0.21192464 -0.09850153 0.21976095 0.16594459 0.18475266 -0.19792043 0.06256042 -0.11703279 -0.012277845 -0.13324524 -0.09877814 0.1922108 -0.1777355 0.22417262 -0.0175469 -0.2147324 -0.13427201 -0.053428102 0.21191214 0.040349625 0.13860646 -0.07468902 0.14227282 -0.12156965 -0.06443877 -0.093755685 -0.085930966 0.10768525 0.08878624 -0.21553957 -0.12831263 0.119905345 -0.12671192 0.12433106 -0.085863054 -0.12778835 0.16462977 -0.15022899 0.034011185 -0.12967359 -0.04940156 -0.22239818 0.22160429 0.14075616 -0.12926574 0.14031325 -0.12578015 0.15402925 0.02221471 0.10313298 -0.17691663 0.06565443 0.12118653 -0.2177742 0.1233458 -0.052539926 -0.10624436 -0.17581953 -0.13164826 0.06994377 -0.10316195 -0.1026312 -0.16469027 -0.018158156 -0.11269389 -0.026726538 -0.13912657 0.23092645 0.16393992 -0.21044189 0.110553585 0.111070335 0.05329293 -0.22327858 -0.18183358 -0.021706155 -0.14601468 0.051987264 0.17622009 0.11393604 -0.17459646 0.102338925 0.11333807 0.2231078 0.053271506 -0.019361285; -0.20664483 -0.05821162 0.22916652 0.21327056 -0.07102652 -0.023487454 -0.11737373 0.07338199 0.15409738 -0.024102626 -0.22450285 -0.15195358 -0.069354996 -0.09231941 -0.14146757 0.11988138 -0.10336607 -0.14247814 -0.06811439 -0.07305257 0.11392907 -0.15359475 -0.13595007 -0.032307107 -0.14206238 0.07447929 -0.22935039 -0.0075826817 -0.19861633 -0.09851859 0.02865722 -0.038407028 -0.23025061 -0.09736539 0.13657671 -0.18266532 0.06976211 0.041736815 -0.06986039 -0.041945826 -0.21253912 -0.07250072 0.12171187 -0.19086449 -0.18540438 0.018375836 0.1409952 0.18925732 -0.14101535 0.10703873 0.19966272 0.12900661 -0.09043166 -0.014839444 -0.13033964 0.1404546 0.22735286 0.14730349 0.17858194 -0.21727386 -0.19258954 -0.09334348 0.1103321 0.20560056 0.06740785 -0.07732594 0.22311912 -0.017162554 -0.05112846 0.17635809 0.06619769 0.1996586 0.10379304 0.090217195 -0.018690877 0.15473498 0.0850573 -0.17859894 0.06629707 -0.032375213 0.0900632 0.14904949 0.18618084 -0.13662417 0.020388857 -0.20294535 0.098324515 -0.17324084 -0.11966342 0.04720635 0.19722603 0.053241048 -0.21713634 0.2177596 0.17049202 0.15343703 0.067914724 0.05592073 -0.072842866], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[-0.67499644 0.34253561; -0.27180654 -0.14506549; -0.6092826 0.06320071; 0.5393801 0.6643296; -0.61429715 -0.066203594; -0.417104 0.6038573; -0.4244111 -0.51997316; 0.6622757 -0.58499175; 0.07892217 0.40343982; -0.6830543 0.28614566], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.23051004 -0.17088094 0.12780623 0.060316898 0.2628151 -0.12407083 -0.27890813 0.16999331 -0.25054052 0.26369593 0.19679976 0.15435652 -0.095946044 0.007997241 -0.26547682 0.12946343 0.23091817 -0.014845921 0.23629211 0.1605042 0.008404895 0.15925677 0.04409583 -0.1494514 0.17018822 0.11004049 0.17648003 0.08663305 -0.21215354 -0.09569194 -0.24099883 0.07636487 -0.22343293 0.27387196 -0.0711237 -0.0706559 -0.08610523 0.1420447 -0.17187017 -0.22226478 0.06739562 -0.13134442 0.017451791 -0.26788372 0.25300866 -0.119625084 0.055896748 -0.057013594 -0.03203756 -0.14191401 -0.20704374 0.06286009 0.114731185 0.11252921 -0.27729332 0.13512866 -0.25226298 0.19671609 -0.090922065 0.2476492 0.1797741 -0.17036211 -0.15589274; -0.23699006 -0.21747088 -0.2464361 -0.25855923 0.23947646 0.1367627 -0.036736827 0.25025088 0.26282 0.15867442 -0.09284831 0.099788405 0.2219672 -0.1774813 0.16386212 -0.03130831 -0.14189322 -0.2772456 0.05284816 -0.15644933 -0.20473018 -0.10153724 -0.13057204 0.023975898 0.05231819 0.009338898 -0.01632589 0.18447872 0.21266478 -0.15856898 -0.13905172 0.21802475 -0.1490892 0.059792567 0.21422024 0.14568335 -0.023992714 -0.21004415 -0.029482136 0.18202704 -0.005674999 -0.24602732 -0.16916488 -0.26647982 0.26557478 0.09071557 0.20812808 -0.024367353 -0.23761986 0.048401352 -0.012097194 -0.27745774 -0.22576036 0.08541791 -0.080281906 -0.03715265 -0.07122971 -0.061473284 0.12272699 -0.046621453 0.16569845 0.12054695 0.21753335; -0.26825723 -0.087400235 0.26763391 0.033851806 0.049916144 -0.07565178 0.27646902 -0.11728179 0.0036576781 -0.21083303 -0.10900687 -0.029280769 0.20360558 0.24636942 -0.26552728 -0.17859301 -0.10460012 0.09153816 0.18033071 -0.28383416 0.051444847 0.20727 -0.003188541 -0.0001425491 0.10533921 -0.1476896 0.22942358 0.06973522 -0.27958784 0.15821636 -0.071664296 0.28208953 -0.0071036345 0.14963678 0.18294829 0.2752055 -0.0043829666 0.13395658 0.25669304 0.0077320333 -0.02368663 -0.11218211 -0.265979 -0.06262239 -0.22973636 0.27592456 -0.019462686 -0.12534681 -0.17847729 -0.1251616 -0.0039728857 -0.19082725 -0.028449329 -0.054254923 -0.07127572 0.03545553 -0.25531128 0.16201736 -0.1362618 -0.043973852 0.014853507 -0.06149943 -0.22586173; 0.061847825 -0.13845496 0.20417444 -0.15874074 -0.0003535874 0.052026834 -0.17116392 -0.2615502 -0.08705919 -0.2849176 -0.119349144 0.15504183 -0.1554014 0.10331006 0.27896586 0.17785293 0.06777231 0.13334784 -0.21490246 -0.22283874 0.046004094 0.103922024 0.061059512 -0.08228699 0.08090302 0.24903056 -0.18192342 0.014332217 0.027824998 -0.111030474 0.20168887 -0.11055013 0.24608108 0.18956082 0.06750816 -0.20508237 -0.07240869 0.07220538 0.08709699 0.25072974 -0.12519304 0.24175976 0.21709463 0.22240466 -0.26974398 -0.24805854 0.1811092 -0.23527345 0.0017193042 0.16158873 0.08117937 0.26582742 -0.2636927 -0.0050638597 -0.104387365 0.20588134 -0.10426403 0.21419929 -0.10146926 -0.19915587 -0.010994873 0.23576717 0.15361883; 0.04984899 0.062893756 -0.1805884 -0.118505746 0.21270001 0.16275027 0.020117743 -0.26641718 0.15523584 0.09455394 0.28069973 0.17378284 -0.009010464 -0.1647194 0.0033889846 -0.18095161 0.2806984 0.267946 -0.1895694 -0.05898758 0.26065257 -0.13867345 -0.26624927 0.15798303 0.24765886 -0.20828623 0.095861144 -0.07331532 -0.0953983 0.28066158 0.2208571 -0.006417307 0.1744272 0.24749638 -0.05355366 0.25807834 0.16090308 -0.07917771 0.14552937 -0.04344785 0.157132 0.26334777 -0.21789056 0.100472204 -0.0982164 -0.0039579165 -0.019019933 0.26600963 -0.1712604 -0.28245276 -0.08499013 0.017382378 0.25956112 -0.059220422 -0.12963665 -0.27180663 -0.19083132 -0.18127677 0.27978405 -0.23209083 0.23888855 -0.13325351 -0.14562087; 0.21655205 -0.012329216 0.16605651 -0.24025342 -0.048223566 -0.17943785 -0.22379468 -0.13491099 -0.09845297 0.08189502 -0.25715432 0.11359585 0.19697118 -0.20728073 -0.26475036 0.24065547 0.09218857 0.11682062 -0.037545677 0.14765108 0.15853962 -0.12332497 0.03384863 -0.12608911 -0.23898363 -0.009616853 -0.26246879 0.17767292 -0.26885548 -0.080085054 0.15293205 0.24521324 0.07589368 -0.22135897 -0.023569679 -0.15336479 0.08831079 0.17023982 0.040640917 0.1459804 0.17528616 0.19991957 0.006803157 -0.20926294 0.24284492 0.21002805 0.080541715 -0.011537968 -0.03166422 -0.023291929 -0.13876224 0.13998808 0.13815106 0.24311619 -0.09883294 -0.036933787 0.10318372 0.124747075 0.16030824 0.0803818 0.0043968763 -0.17637347 -0.09239376; 0.18829979 0.17668591 0.04285328 -0.21735704 0.20939076 0.23240614 -0.0066262265 0.19664425 0.23675984 -0.16480091 0.101442404 -0.19622365 0.04974841 0.12663737 -0.12782629 -0.02583888 0.14200458 -0.26475668 0.13808319 0.0840062 -0.021739474 -0.2476591 0.053587187 -0.18487614 -0.04781383 -0.011118591 -0.021759843 -0.2701421 0.26908672 -0.17070924 0.05579152 0.09870768 0.12696782 -0.17425194 0.1445448 0.10189059 0.116836816 0.2074572 -0.2518963 -0.018348062 0.046255562 -0.12508102 -0.18293706 0.2252345 0.08814429 0.23858695 -0.102691546 -0.27797434 -0.16377385 -0.20959978 0.19958547 0.28363705 -0.043296855 0.0033721356 0.07070507 -0.053595662 -0.19026689 -0.12041367 -0.10377944 -0.1300637 0.0106913885 0.20993173 -0.011673853; -0.1005854 0.022222076 -0.20027284 0.08475465 -0.0013654776 -0.17350824 -0.032671187 -0.08005282 -0.07109359 -0.24576351 0.2617431 0.17758222 0.24515042 -0.003781738 0.27737722 0.002457306 -0.13030455 0.22185118 0.09037173 -0.1431843 0.0774434 0.27426547 -0.16522042 0.28576988 0.21457537 0.08830034 0.17538086 -0.2670287 -0.16906717 0.09718486 -0.22459967 -0.0146215195 0.0015256958 0.041694265 -0.1629552 0.22953062 -0.10614997 -0.12037081 -0.06418104 -0.2225658 -0.042160906 -0.010112067 0.2464024 0.09669597 0.21655577 -0.057800263 -0.23312312 0.2290664 0.0038743215 -0.037651934 -0.08370602 -0.11597083 0.17334063 -0.21027993 -0.03269289 -0.014667383 0.1161807 0.17260127 -0.08418832 -0.02805162 0.0910587 -0.19292666 -0.13742298; 0.103297144 0.20838058 0.07808451 -0.055319376 -0.1393753 -0.1301206 0.2856093 -0.13748805 0.047006518 -0.016344992 -0.25730497 -0.002099891 0.26130486 -0.17825049 0.15934567 -0.2594467 -0.21957402 0.0918249 -0.030839888 0.21615478 -0.057170738 -0.0031236743 0.037601933 0.21432143 -0.19019066 0.004744893 -0.24191281 0.05708803 0.053049456 0.13537031 -0.14410996 -0.24476522 0.11247699 0.061793722 -0.13064271 -0.18640576 -0.11965687 -0.15879461 0.2155596 -0.056149926 -0.084812276 -0.23595701 -0.07765441 -0.27943686 0.081153095 -0.14262046 -0.06420557 -0.008680663 -0.19088614 0.050029714 -0.0024268208 -0.24232696 -0.25448886 0.030460157 0.1293445 -0.2034683 -0.09377776 -0.11447541 -0.27084008 0.24768466 0.26519248 0.21911234 0.2848391; 0.13831292 -0.09032654 0.24129306 0.24591751 -0.1844175 -0.22051547 -0.23799565 0.20073436 0.019648537 0.21984743 -0.14400277 0.16183989 0.11759836 0.20328332 0.107287906 0.2124768 -0.185407 -0.27066985 -0.2362037 0.27495778 -0.20298645 0.20601039 -0.22097042 -0.045490354 0.06705823 -0.27575323 0.031520408 -0.10701573 -0.27298558 0.024281023 -0.16757856 -0.13419783 -0.20551617 -0.071223155 0.076067224 -0.2050335 -0.2260996 0.15651889 0.031992413 -0.10746856 -0.01973032 -0.23726863 0.274604 0.08307732 0.10730722 -0.20218244 0.15770689 -0.1476008 0.19412543 -0.23853736 -0.21048628 0.2820913 0.004421722 0.15349703 -0.05630437 0.108745456 -0.12121968 -0.06340404 0.060741607 -0.21585797 0.05963621 0.21314786 -0.104219906], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[-0.3229558 -0.30737773 -0.18375857; -0.27691472 -0.3213116 0.60526323; 0.24539484 0.14705099 -0.47891378; 0.27015355 0.44633394 -0.654428; 0.48623878 0.29607806 0.41522643; -0.21722521 -0.121872365 -0.045540318; -0.41786978 -0.35949442 0.42454696; -0.49580407 -0.4200282 -0.6166244; -0.653852 -0.4924457 -0.59517473; -0.23072563 -0.44474968 0.3801429], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.19167826 -0.2005456 0.093027115 0.2919471 -0.4525514 0.14941604 0.27724198; 0.24705055 -0.3899697 -0.2383769 -0.2940351 0.1265744 -0.380112 0.047773212; -0.27201015 0.23432729 -0.12393922 -0.38608494 -0.15593751 0.20254296 -0.36166158; 0.23978588 -0.34707046 0.5440005 -0.07208878 0.09178053 0.2319941 -0.1512891; -0.013525512 0.15534721 -0.33925053 -0.21241198 -0.34342572 -0.24226199 -0.04107016; 0.03805241 0.4868758 0.57239217 -0.5742062 -0.56243646 -0.24671458 0.26510122; -0.24717045 -0.37709856 -0.14926888 -0.44597688 0.27541885 -0.5300164 -0.50972503; -0.52920604 0.55722284 -0.3573865 0.14160769 0.17889665 -0.10718824 -0.45319906; 0.0732717 0.51757675 -0.34393924 0.5124288 0.4948161 -0.56787896 -0.28333378; -0.15275908 -0.5922906 -0.0025829081 0.1296202 -0.274112 -0.33697692 -0.35024434], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[-0.4336404 0.39134496 -0.47326693 -0.056723908 0.14990032 -0.44278485 -0.21365114; 0.44550282 0.5747059 0.54858875 0.20945968 0.31732023 -0.42292604 -0.59062135; 0.23441496 0.02329157 -0.04524165 -0.26926288 -0.5357172 -0.10216853 -0.3281726; 0.12209483 -0.3002582 -0.24107885 0.550901 -0.0928342 -0.2619169 -0.18388243; -0.19647141 0.38184872 -0.25967756 0.4450424 -0.06496144 0.04599901 -0.30084622; 0.21024898 0.51694727 -0.32888138 -0.56666654 0.40065685 -0.47655708 -0.08426565; 0.24230002 -0.447869 -0.057576947 -0.07352177 -0.5096584 -0.102979645 0.3625926; 0.31073764 0.29492503 -0.35938832 0.49078852 0.06726326 -0.40312347 0.3283037; -0.19401811 -0.10637196 0.17188524 -0.091331236 -0.4432677 -0.25047147 -0.07945472; -0.52960104 0.10810431 -0.42134774 0.45309126 0.092543975 0.5049 -0.56001407], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[-0.40298754 -0.19131061 0.1016504 -0.13350923; -0.6545591 -0.14159667 0.06485487 -0.560009; 0.46344277 -0.257973 -0.11357925 -0.4621086; 0.4608856 -0.5846482 -0.31752202 0.61994445; -0.07243919 0.34825924 0.2866713 0.20086329; -0.3833766 -0.3519498 0.5105051 0.5229394; -0.13438524 -0.08585971 -0.60809994 -0.18082896; 0.2530726 -0.20759954 -0.2402351 0.14625213; -0.028829832 -0.1468418 -0.2570717 -0.12472808; -0.16986111 -0.17584395 0.27537054 0.08528104], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.3391974 -0.030197544 -0.13834168 -0.025594246 0.20504504 -0.119572096 -0.092744984 -0.20609984 0.2434408 0.20105207 0.01766493 0.17305504 0.033617217 0.29952034 0.15471847 -0.1451842 -0.38168097 -0.11220869 -0.19833086 0.07361042 0.029660486 -0.07903857 -0.054884598 -0.26532772 0.34807697 -0.009103636 0.15453467 -0.13079508 -0.2831765; -0.0947191 -0.33980286 0.023089388 0.3842136 -0.14234783 0.115454935 -0.14814551 -0.32829043 -0.30926985 0.33693784 0.063427 -0.16452044 0.065494254 -0.24742185 0.33155224 -0.1252584 -0.28311774 0.03216511 0.03439134 0.37036404 -0.28994063 -0.34616253 0.17804353 0.020124761 -0.18412974 0.24945235 -0.1694552 -0.2795104 0.38988274; -0.08838347 -0.37396023 -0.27726507 0.046963185 0.06752148 -0.35535243 0.20054474 0.2926541 0.38855305 -0.09739074 0.34239364 -0.0065396763 -0.36037084 0.077351265 0.28652474 0.27688044 -0.060699902 -0.051275276 -0.28163004 0.22591536 0.030389765 -0.010662773 -0.05885503 -0.20731784 0.11167387 0.36620718 -0.13022916 0.3661918 0.35081175; -0.11383066 0.3904362 -0.07801452 -0.2843936 -0.3411165 0.29705444 0.2931338 -0.03674919 -0.13581409 0.087294064 0.2588187 0.040941536 0.009561535 0.044024087 -0.05602025 0.27694455 -0.16537526 -0.06418013 0.04785822 0.06343865 0.13830122 0.19087759 0.18501389 0.028689982 -0.36142468 -0.10538061 -0.056586485 0.25180778 0.014516685; 0.052105132 -0.33981007 -0.00037139666 -0.09604346 -0.22028661 0.32238254 -0.16724946 -0.003428604 -0.019285506 -0.12781371 -0.20717172 0.11914926 -0.040502995 -0.059667446 0.23763798 0.1255583 -0.37751275 0.26869065 -0.3738089 -0.30848244 -0.37378514 -0.25116858 0.22732164 -0.3713533 0.32673198 0.10275717 -0.14351103 0.20194851 -0.016374838; -0.13864289 -0.18477982 -0.024768364 -0.27498746 0.0023075407 -0.36147392 -0.11238454 0.37084544 0.19652653 -0.3638814 0.19958033 -0.3150801 -0.36249408 0.110299475 0.25173324 0.20295398 -0.36562768 0.2903716 -0.13646674 0.21410939 -0.3235486 0.29551083 -0.20739976 0.028016297 -0.221491 -0.31095344 0.12617096 -0.37298492 -0.0397206; 0.2018266 -0.12543383 0.0037681586 0.09182853 -0.028916383 -0.34087643 -0.1967454 -0.2799707 0.18160173 -0.24320191 -0.18050027 -0.29124436 -0.3769101 0.19332138 -0.21888669 0.30859578 0.04526167 -0.27593556 -0.23429592 -0.03733745 -0.05266604 -0.15124442 0.3068688 -0.2845108 -0.18918855 0.37084964 0.060032625 -0.16881776 -0.25421536; 0.34050223 -0.08569696 0.2314965 0.24852464 -0.05376419 0.30885768 -0.03589301 0.26827416 0.025004912 -0.050241087 0.04188553 0.3084969 -0.029199734 0.37600458 0.02569501 0.07947734 -0.2888289 -0.06665385 -0.0874413 -0.1052877 0.22120063 0.015729487 -0.08927837 0.23962869 0.086190164 0.07535892 0.29266477 -0.23879836 0.3890635; 0.26427835 -0.38018537 -0.02523828 0.13411866 -0.32045907 -0.27189255 -0.24801044 -0.3867552 -0.18594141 0.3737637 0.3382588 -0.14686126 -0.15666182 -0.13576832 0.0025671865 0.16302176 -0.2473425 -0.1909632 -0.07821203 -0.3607602 0.14817543 0.32762712 -0.05296122 -0.006121522 -0.24894367 0.20897858 0.046502806 0.11690999 -0.26370883; -0.1791839 -0.3364985 0.3355532 0.06941826 -0.03768888 0.06902577 0.23592754 0.3209204 0.020844877 -0.13871892 -0.26823258 -0.38084528 -0.19671538 0.24705672 0.24816792 0.22618598 0.26679146 0.076907866 -0.25178772 0.08146917 -0.30396792 0.2552407 0.064591594 -0.23718667 -0.19750956 -0.2271961 -0.28695294 -0.3900727 0.36231762], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[-0.2657773 -0.33315572 0.34774923 0.17114742 0.03491177 -0.1051171 0.2065891 0.29601514 0.020180393 0.08397581 -0.3468201 -0.21994883 0.3772128 0.19392519 0.08045101 -0.33723077 -0.30033654 -0.24810834 -0.02121704 -0.16821337 0.31366858 0.069118425 -0.009303932 -0.11427391 0.012353316 -0.029092321 0.38242576 -0.014049612 0.30474505 -0.11257237 -0.11276359; 0.35310167 -0.23830447 -0.32306924 0.36533415 0.30578592 0.25914893 -0.110355385 0.029365528 -0.040632945 -0.06858993 0.049757656 -0.086929105 0.1332337 0.103301644 -0.07345546 -0.30588192 0.32670072 0.38215548 0.31753597 0.35056838 -0.086903974 0.24868235 0.010170162 -0.06996112 -0.33586085 0.19830057 -0.36825073 -0.30402204 0.14768776 -0.16353276 -0.2812391; -0.32763544 0.17450327 -0.15593922 0.03266833 -0.33007437 -0.21157251 0.05429858 -0.30412045 0.24322832 0.24342094 -0.25131574 -0.3015809 -0.3108706 0.21288972 0.3608225 -0.25445566 -0.33486393 -0.07092791 0.3534975 -0.20660006 0.33224162 -0.19576508 0.060784243 0.037399825 -0.35599282 0.08223404 -0.011937736 0.22720934 -0.1369456 -0.14955051 -0.26962677; 0.2604871 0.15371087 0.05895515 0.11981733 0.29605004 0.25058997 0.26305637 -0.28466457 -0.01725318 0.36188674 -0.08623457 -0.3636396 0.091138355 -0.17194133 0.089896314 -0.29315913 -0.07586184 0.21175899 -0.13717394 -0.273673 0.20284623 0.029248284 0.04678124 -0.28521773 -0.05230144 -0.35816726 0.2566447 0.36683044 -0.3567801 0.17073417 0.29793414; -0.16053551 -0.26393634 -0.28212953 -0.25751072 0.18662387 0.026052924 0.085132435 0.36732185 0.012940455 -0.24887338 -0.14627777 0.008574374 0.13516891 -0.0871096 -0.23850124 0.16552918 0.18848415 -0.28865784 0.04006838 -0.16579783 0.25347847 -0.29356056 -0.013343814 0.32646942 -0.3421506 0.1810457 -0.24780472 -0.31004637 0.02469058 0.11147434 0.2935835; 0.30246964 -0.17502451 -0.28407404 0.3270066 0.26710087 -0.20878315 0.37794688 -0.24404995 0.36772656 0.18393794 0.03831011 -0.14440818 0.35762838 -0.14265008 0.15251558 0.15177959 -0.1911454 0.010686524 -0.16544814 0.06358395 0.31152818 0.33484632 0.043292332 -0.33874944 0.2201297 -0.26483718 0.07783636 0.11590651 -0.03501593 -0.305713 -0.15841551; 0.037324626 0.11669097 0.079811566 0.34500676 0.18243514 -0.045683663 0.25144878 -0.38208646 -0.07928179 -0.15854444 0.121312976 0.06768503 0.27097705 -0.14808948 -0.11551491 -0.19908813 0.24991782 0.20605934 -0.32696497 0.093972586 -0.26626304 -0.33380598 0.27124432 -0.03902416 0.31892347 -0.045706466 -0.19136864 0.35799032 0.2848882 -0.19392915 -0.2109311; 0.3283317 -0.009006919 -0.08608517 -0.37188822 -0.16045927 -0.3014493 0.10584483 0.17319523 0.17420201 0.22661823 0.14186098 -0.007188772 0.037044078 0.20558187 0.19359133 0.24624231 -0.053090144 -0.14679906 -0.04233453 0.29085213 -0.21275003 -0.06059043 -0.12993993 0.33866477 -0.09085238 -0.19550014 0.18044092 -0.24103126 0.23630504 0.24420062 0.16675448; 0.21895751 0.3665454 -0.24613924 0.36984926 0.23847339 -0.31188506 0.17351504 -0.0054066963 0.102508694 0.20641956 -0.07574587 0.20106119 -0.0076464894 0.10791744 0.091783226 -0.34365404 0.37664077 -0.0845536 -0.13764073 0.06776698 0.13175 -0.3028246 0.16583933 0.37577766 -0.3747351 -0.17322834 -0.007870537 -0.23190126 0.049782876 0.32834643 -0.2716868; 0.29705966 0.059291247 -0.013541001 0.12251279 0.048467137 -0.24149166 0.2134301 0.10317058 0.23206177 0.09624439 -0.17112608 0.2584991 -0.10523768 0.30830225 0.16027562 0.16110837 -0.3739258 -0.0828869 0.077088654 -0.047486626 0.16037157 -0.095143534 -0.19501856 -0.013173122 -0.109111786 0.08926412 0.15381868 0.10776007 0.25870386 0.3431325 -0.25567102], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.18387623 0.29213774 0.27113417 0.34511113 -0.39515603 0.18484809 -0.07040607 0.16631961 0.40850317 -0.29641977 0.03695208 0.11868787 0.14160717 0.10393915 -0.05730854 -0.32775 -0.3207192 -0.43574637 -0.12396111 -0.111158535 -0.28967613; 0.32983634 0.43082199 0.084580615 0.04297666 -0.073779866 0.09438166 -0.43158683 -0.08498586 -0.057174385 0.3147801 0.21391594 -0.26730362 0.39278907 -0.23400393 -0.3798965 0.25125837 -0.42503124 0.15902989 -0.2849139 0.09212458 -0.223001; 0.08021913 -0.019811083 -0.2602453 -0.12371599 -0.18101226 -0.36277556 -0.017248876 0.37101144 0.19223398 0.047413144 -0.12380436 -0.024725974 -0.17465606 -0.18239266 -0.21481873 -0.24410291 0.05303332 -0.0064423033 -0.273076 0.098525085 -0.38026434; -0.0075370944 -0.3478771 -0.06565848 -0.32314202 0.0038847656 0.15385813 0.3657084 0.16283946 0.3412721 -0.25935102 -0.14418924 -0.3475245 -0.05128779 -0.36149713 -0.40454853 -0.42683554 -0.1889172 -0.31020886 0.35538805 -0.24256875 -0.32673028; -0.20561598 0.29853076 0.068223566 -0.33201852 -0.21337087 -0.11822483 0.01813567 -0.091204174 -0.25782225 0.4049386 -0.03381623 -0.14555565 -0.34109145 -0.1918527 -0.40207788 0.40998214 -0.01082084 0.16432679 0.017044656 -0.37699187 -0.37615085; -0.27515462 0.4262299 -0.35075137 -0.30304873 -0.29709283 0.39324084 0.16930373 -0.16878709 0.2310057 0.2809926 0.21465547 0.020793745 -0.23319444 -0.43228814 0.42361534 -0.013841572 -0.22592513 -0.23437074 0.31902263 -0.031814557 -0.022642225; -0.2073699 0.19160412 -0.34361827 -0.35329288 0.34901232 -0.11455477 -0.013441678 -0.20550165 0.32175392 -0.34809703 0.07340997 0.267614 0.09723253 -0.22180751 0.261098 0.00462765 -0.15797831 -0.26886854 -0.3067364 0.059434455 0.2364871; 0.34576276 0.37775347 -0.26761538 -0.20161085 -0.062385697 0.30365202 -0.20844942 0.11529157 -0.14746617 0.29925156 -0.11010444 -0.17382786 0.2818807 0.052629806 0.35695988 0.025996929 0.027483327 0.43765044 0.22348465 -0.43593228 0.41265625; 0.19411382 0.4192722 0.2217773 0.27639332 0.16110693 0.011401354 0.13030335 -0.406191 -0.28482696 -0.2877185 0.3397525 0.08338529 0.2497212 -0.12641318 -0.35954696 0.21716896 -0.38097602 -0.27852175 -0.38840422 0.31897685 -0.22469871; -0.027911646 0.33953044 -0.43197802 -0.3277015 0.068906926 0.28759927 -0.16228594 -0.20004211 -0.15567702 0.1946773 0.0022238293 0.003362832 -0.41944867 0.023102693 -0.06605397 -0.115440674 0.40675545 0.39955837 -0.239221 -0.12761727 0.04178301], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.24117164 0.23081018 0.23020047 0.34507298 -0.013347041 -0.020850815 0.15353546 -0.3070773 0.3463223 -0.09651567 0.08607916 0.18823442 0.14351355 0.076092504 0.03769828 0.0763608 -0.33506668 -0.14983098 0.2127508 -0.267094 -0.25533932 -0.25305527 0.24977016 0.08182135 0.3334334 -0.36157438 -0.1296597 0.19180924 -0.15481573; -0.24659802 0.2042754 0.24524958 0.08975254 0.018512273 -0.03955587 -0.3161264 -0.100043215 0.049691122 -0.009771617 -0.03385699 -0.31981054 -0.28444898 0.09005076 0.0709607 -0.32778496 0.20599829 0.15972501 0.09664173 -0.09253009 0.35565388 0.03069687 -0.3896988 -0.2382102 0.08933074 0.09807514 -0.36225003 -0.17342053 0.27090457; -0.10701245 0.29205602 0.022500006 -0.1967973 -0.21218185 0.2770404 -0.075696744 0.11257232 -0.21806908 -0.052632418 -0.3669497 0.053282678 0.1698928 -0.30094364 0.030736988 0.33984885 0.11816216 -0.20094383 0.2934299 -0.1930088 -0.27835616 0.27422175 -0.29868516 -0.25953576 -0.11336538 -0.1668163 0.10884592 -0.102739215 -0.35136923; 0.09995185 0.3740238 0.09093331 0.24295916 0.28177685 -0.08076122 -0.1785638 -0.027987167 0.21452989 -0.32368898 -0.3346672 -0.19106938 -0.060887028 0.24925199 0.31332546 -0.17298199 0.053892307 -0.02549423 0.09734973 -0.17406294 -0.359173 0.3095548 -0.24649371 0.022688674 0.23143922 -0.20671925 0.26358718 0.27954292 -0.3909742; 0.044509012 0.32798532 -0.10594137 -0.25663814 -0.21241653 -0.15894112 0.10908971 0.2244725 0.071865276 -0.35063177 -0.143507 -0.32612067 -0.074408054 -0.18268703 0.32545558 -0.38092068 0.24808338 0.22928363 -0.03150293 0.19530012 0.28829437 0.13678478 -0.06768766 -0.2398067 0.28186354 -0.36230835 0.3754365 0.15221694 0.039020166; -0.24294293 -0.011803708 -0.36900663 0.22918418 0.2558495 -0.19832394 -0.07170527 0.018208068 0.34952796 0.19680403 -0.15465999 0.11854034 -0.03296116 0.10263107 -0.33809257 -0.18315545 -0.34227747 -0.041029114 0.33557522 -0.0610706 -0.25960845 0.15409061 -0.13662034 0.22102702 -0.25788474 0.14437193 0.12616195 0.012529061 0.23875464; 0.39099997 0.11529044 0.3225645 0.3234444 -0.2461182 -0.09365882 -0.15244123 0.3249415 -0.0016508752 -0.024991024 -0.07935801 -0.33862093 -0.34540707 0.109269686 -0.21480848 0.08751467 -0.02304436 0.1370536 0.38683155 -0.24609445 0.0154467905 0.26219457 -0.0038068742 0.13044453 0.063598745 -0.12062798 -0.021214169 0.029900633 -0.071669966; -0.05993761 -0.029199688 0.10454084 -0.3482234 0.043384954 0.15863363 -0.13256915 0.10845203 0.072859764 0.32101506 0.11337852 -0.3482073 0.15855911 0.116973065 0.13766538 -0.08439237 0.22298312 -0.024832657 -0.0795472 0.2539572 -0.35659096 0.24951795 0.058599968 0.31251192 0.32299358 0.09341011 0.31361246 -0.32447138 -0.17206256; 0.15531501 -0.06817011 -0.3834824 0.27014923 -0.33172 0.22038512 0.03779161 -0.063341014 -0.24196798 0.14497899 0.1684538 -0.1878214 -0.013673644 0.033644572 -0.12545094 0.21755394 0.052252464 -0.16141647 -0.24646117 -0.38751802 -0.30511242 -0.009933212 0.15035051 -0.33847687 -0.25275245 -0.14473164 -0.1521468 -0.18188855 -0.28689748; -0.17784896 0.22801954 -0.043042127 0.19663496 -0.1373094 0.1300303 0.22328293 0.11200931 -0.15121333 -0.09825604 -0.22233291 -0.05199254 -0.2882172 0.032113444 -0.010349637 0.24332583 -0.25216818 0.122266084 -0.29580867 -0.102137536 -0.06862361 0.30719063 0.29821575 -0.3689593 -0.056586392 0.32337222 0.07050397 -0.35758072 0.019520137], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.32194614 0.02380971 -0.22501437 -0.36032766 0.15566674 -0.297637 -0.24393909 0.12982045 -0.35635382 0.34450707 -0.090162225 0.024492297 0.3748882 -0.078394495 -0.13791645 -0.24934126 -0.37850198 -0.23427603 -0.3193447 0.04026885 0.37654284 0.321495 0.3575722 -0.16428298 -0.07784489 -0.31074485 0.30270922 0.018100165 0.10779231 -0.21922015 0.22129102; 0.10493277 -0.19520043 0.12911953 0.36049256 0.27198997 -0.31387398 -0.07124371 0.052522477 0.20773008 -0.2741768 -0.36049625 -0.36611328 -0.057614923 0.27412373 0.16360623 0.32142434 -0.31787273 -0.2657675 -0.019763673 -0.1743152 -0.1238926 -0.041251868 0.13010657 0.37343383 -0.08701775 -0.36908376 -0.31991625 -0.03028849 -0.2908752 0.28516164 0.073432155; -0.11297377 0.15826078 0.33343196 0.04864198 -0.14663948 -0.16935787 -0.043439947 -0.16519709 0.3603967 0.11164508 -0.25024694 -0.020123389 0.2512338 0.09955868 -0.24670263 -0.09243854 0.01197513 -0.13748774 -0.026419345 0.25536433 0.22802523 -0.15958911 -0.34105316 0.37446436 0.28808743 0.2358513 0.08181764 -0.16074187 -0.16120136 -0.067048416 0.037819784; 0.032144625 0.3310267 -0.1044206 0.31924102 0.21809307 -0.35615262 0.17239207 -0.31631052 -0.23340306 0.27525377 0.18746756 -0.023316149 -0.093683094 0.31575397 -0.10961046 0.18323013 0.050893217 0.13099317 0.1658266 0.10254632 0.25599954 -0.058145516 -0.33983645 -0.15178533 0.1481312 -0.23345208 -0.25571594 0.18555246 0.29681158 0.21766795 -0.06981182; 0.20103945 0.054610185 -0.11852403 -0.29863337 0.33262604 -0.059005544 -0.20107971 -0.073055245 0.35409436 0.1426053 -0.09240845 0.353484 0.044165675 0.13793911 -0.35379267 0.12527679 0.3476704 -0.2674968 -0.37635267 -0.2281344 -0.19972762 -0.18844904 -0.09059144 -0.33873448 0.17717186 -0.0472992 -0.11254478 -0.17235243 0.020634234 0.3649633 0.13211049; 0.041391186 -0.21652496 -0.13840097 -0.27892095 -0.15146096 0.08327785 -0.3515314 0.17915331 0.32422578 0.16136822 -0.26458034 -0.35970098 0.27379677 -0.24814296 0.1577679 0.05014893 0.16742411 0.32470593 -0.02213088 0.37116402 0.31179786 0.27083164 0.20093451 0.11717897 -0.098262735 0.30570415 -0.060793594 0.08445386 -0.08221197 -0.2163695 -0.05867492; -0.19071993 -0.25969958 -0.28932112 -0.27476823 0.07497285 0.14347112 0.018381216 0.051128756 -0.19781297 0.34888896 -0.057928443 -0.1595192 0.26423037 0.122686446 0.25656196 0.05365033 -0.17111386 -0.1923092 -0.30673498 -0.060150314 -0.29093415 -0.06819501 -0.22598134 -0.3582016 0.34429833 -0.16961786 0.30163518 0.32822075 0.045092694 -0.029632352 -0.20482239; -0.042324636 0.23369387 0.356124 -0.3444401 -0.060261633 -0.034081794 -0.18290581 0.3055088 -0.006386751 -0.19471358 0.20091048 0.35772645 -0.3150698 -0.30110762 0.3421879 -0.18495794 0.05520024 0.29844028 0.12154573 0.11288836 0.20142962 0.1593029 0.37828234 -0.32862365 0.26551613 0.20596512 -0.21350138 0.26182693 -0.13261431 -0.10138354 0.345537; 0.0933797 -0.043870624 -0.07038263 -0.011624077 -0.051332466 0.338141 -0.18900959 -0.05455788 0.15821582 0.35535562 0.3815319 0.35870272 -0.095762186 -0.33861938 0.021429505 -0.07139908 0.3427599 0.2967601 0.19617824 0.3345417 -0.029888641 0.13151693 0.13143885 -0.13661033 0.24028906 -0.1282698 0.02262686 -0.26384613 -0.19027855 0.03009568 0.26859397; 0.14253348 -0.3622154 0.040966988 0.32630506 0.32700264 0.3415182 0.005561838 0.085153505 -0.12972039 0.12767929 0.1344928 -0.19511192 -0.36018446 -0.08069489 -0.29842204 -0.38022628 0.2014147 0.35982415 -0.3221851 -0.2566148 -0.058875117 -0.15206607 -0.15605715 -0.25619745 -0.0015178059 -0.047065575 -0.2553779 -0.17198548 -0.26722896 0.33595622 -0.30300632], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[-0.2870426 0.40116444 0.08956081 0.039807662 0.15528621 0.042076964 0.3016175 -0.1706395 0.08583081 0.25361443 0.050941598 0.33829966 0.23029491 0.12505391 -0.013226443 -0.16689377 0.26096553 -0.40879592 -0.08371569 0.34470332 0.05118615; 0.24154848 0.052357353 0.23326404 0.19517285 -0.10904426 0.24734418 -0.40102175 0.32164198 0.43918875 0.33883545 -0.0886513 -0.31731114 0.4382465 -0.25380474 0.3660351 -0.15432762 0.42614797 -0.010314849 0.21810965 0.217508 0.087220915; -0.22326915 0.04425506 -0.40836433 -0.43354183 0.34074327 0.26369372 0.073627986 0.399125 -0.42903093 0.10642342 -0.036043417 0.2527251 0.1775803 -0.42815584 0.3848986 -0.1524463 -0.27865082 0.0055508413 -0.15734169 -0.12682776 -0.4063459; 0.017575873 0.009077984 0.16848086 -0.068094395 -0.110608384 0.1636105 -0.0862115 -0.3473899 -0.3675222 -0.32874778 -0.011497328 0.10598142 -0.2902803 0.11589443 0.42842644 0.37817323 0.13225237 0.19801663 -0.28509116 -0.17559457 0.20737262; -0.104861245 0.034512807 -0.22756127 0.16811648 0.121952206 0.37763312 -0.30585015 0.06619636 0.21012925 -0.08586426 -0.21863332 0.22820267 -0.039079934 -0.40450945 0.07584458 0.43638787 -0.39751196 0.36419064 0.34040204 0.0420467 0.14384703; 0.30760086 -0.23023166 -0.0025188331 -0.30647 0.2731295 -0.0035177022 0.10248223 0.41074973 -0.11181179 -0.3450227 -0.05425613 -0.046190914 0.31479812 -0.072312035 -0.23253694 0.33471644 0.20877501 0.22235881 0.43705592 0.24138306 0.10497882; 0.4261311 -0.21265295 0.092790425 0.34332165 0.16648139 0.26273593 0.19408335 -0.17035724 -0.12550284 0.26053402 0.16910155 0.28542808 0.17477119 -0.11355916 -0.13739923 -0.20497558 -0.014417314 0.21846953 -0.34067017 0.27891853 0.28570336; 0.055864412 -0.09827052 0.37853715 0.25717014 0.18522811 -0.35265937 -0.43370205 -0.4175773 0.16896121 -0.09451293 0.30405155 0.0011541068 -0.38567573 0.12817398 -0.42686704 0.3380473 -0.32028487 -0.08710165 0.1784918 -0.044962596 -0.10632556; -0.32414484 -0.07168473 -0.401091 0.35562286 -0.28146884 0.01334649 0.3276865 -0.42849994 -0.15900844 -0.06571297 0.35420704 0.12095549 0.17738421 0.24229744 0.2875925 -0.24264012 -0.39641744 0.09475701 0.18920009 0.11602303 -0.074785605; 0.07979547 0.40427086 0.013089666 0.27722982 0.34899658 0.18403493 -0.3163598 -0.21352533 -0.14277034 0.091853805 -0.057671722 0.3908306 0.31190494 0.43186623 0.11063335 0.12420714 -0.3689055 -0.16281392 -0.41669753 0.32737082 0.081703216], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], Float32[-0.18761852 -0.052535694 0.0045574512 0.22497292 -0.111436464 -0.08278927 -0.24199721 0.27164423 0.048021935 0.0039348053 -0.19183347 -0.21199492 -0.31030956 0.23886453 -0.016440919 -0.28719893 -0.11047198 -0.051914405 -0.27101493 -0.17248021 -0.09946369 -0.117937356 0.24533401 0.24761905 -0.24991402 -0.06544283 -0.11828575 0.25055712 -0.21721965 0.075461864 0.24004678 0.16537404 -0.12688383 0.19582221 0.08104981 -0.2314832 0.30425376 0.11037822 0.11140268 -0.1131236 0.18652275 0.22759461 -0.2423577 -0.21302304 0.2211852 0.088460155 0.2905272 0.098448314 0.037683677 -0.188897; 0.28486097 -0.12169943 0.25768846 -0.083364986 -0.24318339 -0.17375238 0.104363434 -0.30723086 0.15000357 -0.23444234 -0.031826463 -0.11300497 0.07104212 -0.10065443 0.037941605 0.20101821 0.15820405 -0.017900784 -0.042564534 0.06799807 0.27138618 0.30581033 0.17845021 -0.1210763 -0.26190454 -0.28274384 -0.23858918 0.28318846 -0.018996341 -0.20478942 0.26093015 -0.26902777 -0.17823248 0.1581143 0.30644605 0.032155447 0.13251536 0.06852115 0.11990203 -0.112497486 0.04101646 -0.08615059 0.2474977 -0.3000902 -0.23940948 0.20371576 0.20057796 -0.28178334 0.17326465 -0.048196774; 0.14914069 0.14670585 -0.015446652 -0.116284184 0.041281883 0.0071055614 -0.1062253 0.11440433 -0.2740604 0.09227414 -0.1912688 -0.045827158 0.16112284 0.11181162 -0.29875427 -0.108770095 -0.041481905 0.048241407 -0.27756438 -0.08441218 0.21959288 0.266312 0.2392395 0.31213123 -0.10577056 -0.18673536 0.083306216 0.087203175 -0.15281074 -0.08273099 0.22004563 0.07370038 0.021620562 -0.17784415 0.17591228 -0.3030233 -0.048594706 -0.06778896 -0.22242557 -0.0151013825 -0.19828728 -0.26414752 0.3147847 0.21113236 0.18142095 -0.3126612 -0.31456342 0.18144651 0.24431197 0.3130038; -0.07342059 0.19621573 -0.069334775 0.13800906 -0.02106287 -0.15501863 -0.1928503 -0.26985568 -0.07879532 -0.057272885 -0.13090266 0.055047236 -0.2112982 0.11421637 -0.17000881 0.13243061 0.3070214 -0.09134924 -0.24594249 0.011185237 0.2784737 -0.17633362 -0.15088958 -0.20722885 0.1959088 0.20759617 -0.08670942 0.1615121 -0.04625578 -0.00799895 -0.26223108 -0.14788266 -0.01953221 0.13080047 -0.15989138 0.17604487 -0.0937714 -0.030410402 -0.19610445 0.013888472 0.24987376 -0.25276184 0.1376674 0.08271414 0.07816913 -0.16051859 0.27253506 0.3081096 0.14031005 -0.09712804; -0.28071636 -0.24880941 0.09803138 0.009896367 0.24410693 -0.16587919 0.24220495 -0.18663734 -0.0052999747 -0.22532347 0.10025695 -0.023790946 0.27888933 -0.08587668 -0.07731623 -0.16370608 0.30664185 0.27938405 0.12752627 0.0040351176 0.07264697 -0.08879581 0.22512922 0.2769331 0.2576174 0.2627165 -0.018798469 0.16079038 -0.038407166 0.14928314 0.09590925 0.06397546 -0.022286184 -0.052180134 -0.037175708 -0.111218944 0.24139394 0.05724088 -0.12584576 0.2550954 -0.23268232 0.05703147 0.016079701 0.08974318 0.27270362 0.3118786 -0.28198916 0.14490551 -0.11536433 -0.039238542; -0.21618007 0.0067844186 0.13326205 0.30043057 0.26320058 0.102281526 0.14359802 0.17069441 0.06590707 0.05109087 -0.080432326 -0.2735308 -0.28710663 0.0019304404 -0.24037577 0.063144274 0.17988832 -0.04556961 -0.20859326 0.22845803 -0.15843646 -0.2951131 -0.017667625 -0.31289667 0.06827016 0.29621032 0.20056498 0.13080631 -0.09368451 0.27318606 -0.20493923 0.22860852 -0.16404845 0.050472595 -0.17177643 0.13690904 0.029014735 -0.19224791 0.050070893 -0.21164513 0.2046614 0.24119964 0.15863226 -0.053365108 0.28342435 0.12569237 0.31557256 -0.23390156 -0.08811888 0.1432875; 0.22667432 -0.026528975 -0.13344511 -0.009095262 0.10796239 -0.24194847 0.015485932 -0.12029988 -0.09658053 0.25415438 0.16111967 -0.084199265 -0.121348135 0.17489249 -0.10094579 -0.14576836 -0.24340987 -0.2772468 -0.124077074 0.20629023 0.043738317 0.1600703 0.0066329506 0.014307516 0.24471545 -0.011871517 0.11649472 -0.061041296 -0.06395322 0.08084236 0.12290586 0.24088714 0.08745518 0.23536848 -0.08039078 -0.23828018 0.038085833 -0.29668593 -0.0021409043 -0.1954613 0.23526116 -0.009679796 0.024597593 -0.2776047 -0.09459731 0.24198213 0.016321342 -0.13159418 0.21266963 -0.30724055; 0.2669016 -0.1598483 0.29249328 0.19198191 0.30493575 -0.24245375 0.091750525 -0.18476753 -0.089809604 0.1529587 -0.08756186 -0.19301905 0.22358589 0.11155321 0.0018100352 -0.187034 -0.20228738 0.10567809 -0.2881814 -0.12926 -0.1680153 -0.047673725 -0.07178242 0.033855442 -0.018711915 0.16270146 -0.31494737 0.0024156799 -0.03708335 -0.08429935 0.27556106 0.06526674 -0.13904199 -0.017100621 0.07817294 -0.30075735 -0.106721096 0.2308203 -0.16536295 0.12921834 -0.2595945 -0.2847546 0.12741812 -0.30438665 0.24974328 -0.1992584 0.16362254 0.18401843 0.2924148 -0.23765124; 0.31074402 -0.094694115 -0.014838971 0.31381837 0.21834393 0.16682422 -0.13008659 -0.114216484 -0.2225454 -0.18696576 0.28453776 -0.2204235 0.0792651 -0.0626956 -0.29124025 -0.18036447 -0.276397 0.014096636 0.20833458 -0.22381651 -0.07092515 0.25401348 0.25245157 0.22304079 -0.2933343 0.14712267 0.0848653 0.060411524 -0.21958809 0.0035989224 0.110468656 0.23413607 0.16512999 -0.18223132 0.26166636 0.2584796 0.051877197 -0.124807954 -0.2898065 -0.081236295 0.22733893 -0.008287597 0.1422834 -0.25383833 0.13091555 0.23349771 -0.28327826 -0.025150951 -0.123288184 -0.027015157; -0.20909216 -0.22418784 -0.19439164 -0.124106854 -0.12144992 -0.26550457 0.005087701 0.24449344 0.14194635 -0.18293385 -0.16599578 0.015260729 -0.240207 -0.20770033 -0.21506275 0.16361776 0.029939035 0.18481283 0.21695007 0.27632684 0.1700802 -0.0716849 0.24631234 0.28281692 -0.18711068 0.21542114 -0.009494514 -0.20955296 -0.10551056 -0.13941728 -0.0172093 0.03867919 0.07009995 0.274379 -0.22836542 -0.12848076 -0.24079621 0.2974726 -0.184321 -0.05464825 -0.15316848 -0.10142489 0.1481808 0.13654594 0.22446081 0.056623023 0.19530806 0.117109485 -0.015082496 0.13912281], Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]])Lastly we turn our training data to minibatches, and we can start training
data_loader = Flux.DataLoader((ds_train, y_train), batchsize=BATCH_SIZE, shuffle=true)10-element DataLoader(::Tuple{Vector{ProductNode{NamedTuple{(:lumo, :inda, :logp, :ind1, :atoms), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bonds, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bond_type, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}}}, Nothing}}, Vector{Int64}}, shuffle=true, batchsize=10)
with first element:
(10-element Vector{ProductNode{NamedTuple{(:lumo, :inda, :logp, :ind1, :atoms), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bonds, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bond_type, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}}}, Nothing}}, 10-element Vector{Int64},)We can see the accuracy rising and obtaining over 80% quite quickly
for i in 1:10
@info "Epoch $i"
Flux.Optimise.train!(loss, ps, data_loader, opt)
@show accuracy(ds_train, y_train)
end[ Info: Epoch 1
accuracy(ds_train, y_train) = 0.39
[ Info: Epoch 2
accuracy(ds_train, y_train) = 0.39
[ Info: Epoch 3
accuracy(ds_train, y_train) = 0.34
[ Info: Epoch 4
accuracy(ds_train, y_train) = 0.59
[ Info: Epoch 5
accuracy(ds_train, y_train) = 0.61
[ Info: Epoch 6
accuracy(ds_train, y_train) = 0.61
[ Info: Epoch 7
accuracy(ds_train, y_train) = 0.61
[ Info: Epoch 8
accuracy(ds_train, y_train) = 0.61
[ Info: Epoch 9
accuracy(ds_train, y_train) = 0.61
[ Info: Epoch 10
accuracy(ds_train, y_train) = 0.61
Classify test set
The Last part is inference and evaluation on test data.
dataset_test = MLDatasets.Mutagenesis(split=:test);
x_test = dataset_test.features
y_test = dataset_test.targets
ds_test = extractor.(x_test)44-element Vector{ProductNode{NamedTuple{(:lumo, :inda, :logp, :ind1, :atoms), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bonds, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, BagNode{ProductNode{NamedTuple{(:element, :bond_type, :charge, :atom_type), Tuple{ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}, ArrayNode{Matrix{Float32}, Nothing}, ArrayNode{OneHotMatrix{UInt32, Vector{UInt32}}, Nothing}}}, Nothing}, AlignedBags{Int64}, Nothing}}}, Nothing}}:
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNode
ProductNodewe see that the test set accuracy is also over 80%
@show accuracy(ds_test, y_test)
probs = softmax(model(ds_test))
o = Flux.onecold(probs)
mean(o .== y_test .+ 1)0.6818181818181818pred_classes contains the predictions for our test set. we see the accuracy is around 75% on test set predicted classes for test set
o44-element Vector{Int64}:
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2Ground truth classes for test set
y_test .+ 144-element Vector{Int64}:
2
2
2
1
2
2
1
1
2
2
2
1
1
2
2
1
2
2
1
1
1
2
2
2
2
2
2
1
2
2
2
1
1
1
2
2
2
2
2
2
2
2
1
2probabilities for test set
probs2×44 Matrix{Float32}:
0.346156 0.348281 0.346863 0.353872 0.357072 0.348715 0.344788 0.336177 0.345915 0.345533 0.346622 0.348019 0.354409 0.346724 0.346259 0.336045 0.345113 0.347214 0.351089 0.336019 0.340851 0.342078 0.346539 0.325408 0.327077 0.323476 0.354072 0.342912 0.391918 0.398457 0.350724 0.351342 0.340165 0.353031 0.345547 0.352617 0.334764 0.345546 0.32557 0.348303 0.323376 0.346685 0.342451 0.336369
0.653844 0.651719 0.653137 0.646128 0.642928 0.651285 0.655212 0.663823 0.654085 0.654467 0.653378 0.651981 0.645591 0.653276 0.653741 0.663955 0.654887 0.652786 0.648911 0.663981 0.659149 0.657922 0.653461 0.674592 0.672923 0.676524 0.645929 0.657088 0.608082 0.601543 0.649276 0.648658 0.659835 0.646969 0.654453 0.647384 0.665236 0.654455 0.67443 0.651697 0.676625 0.653315 0.657549 0.663631We can look at individual samples. For instance, some sample from test set is
ds_test[2]ProductNode # 1 obs, 176 bytes
├─── lumo: ArrayNode(99×1 OneHotArray with Bool elements) # 1 obs, 76 bytes
├─── inda: ArrayNode(2×1 OneHotArray with Bool elements) # 1 obs, 76 bytes
├─── logp: ArrayNode(63×1 OneHotArray with Bool elements) # 1 obs, 76 bytes
├─── ind1: ArrayNode(3×1 OneHotArray with Bool elements) # 1 obs, 76 bytes
╰── atoms: BagNode # 1 obs, 176 bytes
╰── ProductNode # 24 obs, 104 bytes
├──── element: ArrayNode(7×24 OneHotArray with Bool elements) # 24 obs, 168 bytes
├────── bonds: BagNode # 24 obs, 496 bytes
│ ╰── ProductNode # 50 obs, 56 bytes
│ ┊
├───── charge: ArrayNode(1×24 Array with Float32 elements) # 24 obs, 144 bytes
╰── atom_type: ArrayNode(29×24 OneHotArray with Bool elements) # 24 obs, 168 bytesand the corresponding classification is
y_test[2] + 12if you want to see the probability distribution, it can be obtained by applying softmax to the output of the network.
softmax(model(ds_test[2]))2×1 Matrix{Float32}:
0.34828082
0.65171915so we can see that the probability that given sample belongs to the first class is > 60%.