Conclusion

Chapter V: Noise Subtraction with Neural Networks

5.5 Conclusion

We are actively working on both improving the success on the existing mock data described here and on developing even more realistic mock data. In this work, we present several mock data sets and deep neural networks that have success subtracting noise. In each iteration of mock data, successful networks are designed with the new physics of the iteration in mind.

Figure 5.7: Schematic diagram of network used to solve the Colored Bilinear Mockdata with 16 pairs

10⁰ 10¹ 10² Frequency [Hz]

10 ² 10 ¹ 10⁰ 10¹ 10² 10³

Displacement [am/Hz]

TruePrediction Residual Ideal Residual

10⁰ 10¹ 10²

Frequency [Hz]

10 ² 10 ¹ 10⁰ 10¹ 10² 10³

Displacement [am/Hz]

TruePrediction Residual Ideal Residual

10⁰ 10¹ 10²

Frequency [Hz]

10 ² 10 ¹ 10⁰ 10¹ 10² 10³

Displacement [am/Hz]

TruePrediction Residual Ideal Residual

Figure 5.8: Amplitude Spectral densities illustrating subtraction for the colored bilinear mock data with 1 (top panel), 2 (middle panel), and 4 (bottom panel) pairs.

10⁰ 10¹ 10² Frequency [Hz]

10 ² 10 ¹ 10⁰ 10¹ 10² 10³

Displacement [am/Hz]

TruePrediction Residual Ideal Residual

10⁰ 10¹ 10²

Frequency [Hz]

10 ² 10 ¹ 10⁰ 10¹ 10² 10³

Displacement [am/Hz]

TruePrediction Residual Ideal Residual

10⁰ 10¹ 10²

Frequency [Hz]

10 ² 10 ¹ 10⁰ 10¹ 10² 10³

Displacement [am/Hz]

TruePrediction Residual Ideal Residual

Figure 5.9: Amplitude Spectral densities illustrating subtraction for the colored bilinear mock data with 8 (top panel), 16 (middle panel), and 32 (bottom panel) pairs.

Between 10 and 20 Hz, we achieve roughly a factor of 10 reduction between the target/neural network prediction and the residual between the target and prediction

0 5000 10000 15000 20000 25000 Epochs (training iterations)

10 ³ 10 ² 10 ¹ 10⁰ 10¹

Loss (aka Residual)

act fun = selu num epochs = 25072

# of pairs = 1 Nwhite = 2 Train Dur = 256 s Learn Rate = 0.00000 fsample= 256 Hz Training Loss for bilinear filter

Validation Training Loss Learning Rate x100

0 5000 10000 15000 20000 25000 30000

Epochs (training iterations) 10 ³

10 ² 10 ¹ 10⁰ 10¹

Loss (aka Residual)

act fun = selu num epochs = 30001

# of pairs = 2 Nwhite = 2 Train Dur = 256 s Learn Rate = 0.00000 fsample= 256 Hz Training Loss for bilinear filter

Validation Training Loss Learning Rate x100

0 10000 20000 30000 40000 50000

Epochs (training iterations) 10 ³

10 ² 10 ¹ 10⁰ 10¹

Loss (aka Residual)

act fun = selu num epochs = 53874

# of pairs = 4 Nwhite = 2 Train Dur = 256 s Learn Rate = 0.00000 fsample= 256 Hz Training Loss for bilinear filter

Validation Training Loss Learning Rate x100

Figure 5.10: The loss plotted versus epoch illustrating successful learning for the colored bilinear mock data with 2 (top panel) , 4 (middle panel), and 8 (bottom panel) pairs.

0 20000 40000 60000 80000 100000 Epochs (training iterations)

10 ³ 10 ² 10 ¹ 10⁰ 10¹

Loss (aka Residual)

act fun = selu num epochs = 99999

# of pairs = 8 Nwhite = 2 Train Dur = 256 s Learn Rate = 0.00000 fsample= 256 Hz Training Loss for bilinear filter

Validation Training Loss Learning Rate x100

0 20000 40000 60000 80000 100000

Epochs (training iterations) 10 ³

10 ² 10 ¹ 10⁰ 10¹ 10²

Loss (aka Residual)

act fun = selu num epochs = 99999

# of pairs = 16 Nwhite = 2 Train Dur = 256 s Learn Rate = 0.00000 fsample= 256 Hz Training Loss for bilinear filter

Validation Training Loss Learning Rate x100

0 250 500 750 1000 1250 1500 1750

Epochs (training iterations) 10 ³

10 ² 10 ¹ 10⁰ 10¹ 10²

Loss (aka Residual)

act fun = selu num epochs = 1732

# of pairs = 32 Nwhite = 2 Train Dur = 256 s Learn Rate = 0.00000 fsample= 256 Hz Training Loss for bilinear filter

Validation Training Loss Learning Rate x100

Figure 5.11: The loss plotted versus epoch illustrating successful learning for the colored bilinear mock data with 2 (top panel) , 4 (middle panel), and 8 (bottom panel) pairs.

10 ¹ 10 ⁰ 10 ¹ 10 ²

Frequency [Hz]

10 ⁴ 10 ³ 10 ² 10 ¹ 10 ⁰

DARM [m/rHz]

Amplitude Spectral Density

Subtraction Target

Prediction

Figure 5.12: Amplitude Spectral densities illustrating subtraction for the most bilinear IFO mock data. Below 20 Hz, we achieve roughly a factor of 3 reduction between the target/neural network prediction and the residual between the target and prediction References

[1] D. V. Martynov et al. “Sensitivity of the Advanced LIGO detectors at the beginning of gravitational wave astronomy”. In: PRD93.11, 112004 (June 2016), p. 112004. doi: 10 . 1103 / PhysRevD . 93 . 112004. arXiv: 1604 . 00439 [astro-ph.IM].

[2] Hang Yu. “Astrophysical signatures of neutron stars in compact binaries and experimental improvements on gravitational-wave detectors”. PhD thesis.

Massachusetts Institute of Technology, 2019. url:https://hdl.handle.

net/1721.1/123343.

[3] Denis V. Martynov. “Lock Acquisition and Sensitivity Analysis of Advanced LIGO Interferometers”. PhD thesis. California Institute of Technology, 2015.

doi: 10 . 7907 / Z9Q81B1F. url: http : / / resolver . caltech . edu / CaltechTHESIS:05282015-142013480.

[4] J. C. Driggers et al. “Improving astrophysical parameter estimation via offline noise subtraction for Advanced LIGO”. In:PRD99.4, 042001 (Feb. 2019), p. 042001. doi: 10 . 1103 / PhysRevD . 99 . 042001. arXiv: 1806 . 00532 [astro-ph.IM].

Figure 5.13: Network schematic for successful network on bilinear IFO mock data.

[5] Navdeep Kumar, Nirmal Kaur, and Deepti Gupta. “Major Convolutional Neural Networks in Image Classification: A Survey”. In:Proceedings of International Conference on IoT Inclusive Life (ICIIL 2019), NITTTR Chandigarh, India.

Ed. by Maitreyee Dutta et al. Singapore: Springer Singapore, 2020, pp. 243–

258. isbn: 978-981-15-3020-3.

[6] Ehsan Fathi and Babak Maleki Shoja. “Chapter 9 - Deep Neural Networks for Natural Language Processing”. In:Computational Analysis and Understanding of Natural Languages: Principles, Methods and Applications. Ed. by Venkat N.

Gudivada and C.R. Rao. Vol. 38. Handbook of Statistics. Elsevier, 2018, pp. 229–316. doi: https : / / doi . org / 10 . 1016 / bs . host . 2018 . 07 . 006. url:http://www.sciencedirect.com/science/article/pii/

S016971611830021X.

[7] Sima Siami-Namini and Akbar Siami Namin. Forecasting Economics and Financial Time Series: ARIMA vs. LSTM. 2018. arXiv:1803.06386 [cs.LG].

[8] Katherine L. Dooley et al. “Angular control of optical cavities in a radiation- pressure-dominated regime: the Enhanced LIGO case”. In:J. Opt. Soc. Am. A 30.12 (Dec. 2013), pp. 2618–2626. doi:10.1364/JOSAA.30.002618. url:

http://josaa.osa.org/abstract.cfm?URI=josaa-30-12-2618.

[9] John A. Sidles and Daniel Sigg. “Optical torques in suspended Fabry–Perot interferometers”. In: Physics Letters A 354.3 (2006), pp. 167–172. issn:

0375-9601. doi: https://doi.org/10.1016/j.physleta.2006.01.

051. url:http://www.sciencedirect.com/science/article/pii/

S0375960106001381.

[10] and J Aasi et al. “Advanced LIGO”. In:Classical and Quantum Gravity32.7 (Mar. 2015), p. 074001. doi: 10.1088/0264- 9381/32/7/074001. url:

https://doi.org/10.1088%2F0264-9381%2F32%2F7%2F074001.

[11] M.A. Nielsen. Neural Networks and Deep Learning. Determination Press, 2015. url:http://neuralnetworksanddeeplearning.com/.

[12] Yoshua Bengio. “Practical recommendations for gradient-based training of deep architectures”. In:CoRRabs/1206.5533 (2012). arXiv:1206.5533. url:

http://arxiv.org/abs/1206.5533.

[13] Geoffrey E. Hinton. “A Practical Guide to Training Restricted Boltzmann Machines”. In:Neural Networks: Tricks of the Trade: Second Edition. Ed. by Grégoire Montavon, Geneviève B. Orr, and Klaus-Robert Müller. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 599–619. isbn: 978-3- 642-35289-8. doi: 10 . 1007 / 978 - 3 - 642 - 35289 - 8 _ 32. url: https : //doi.org/10.1007/978-3-642-35289-8_32.

[14] Nitish Srivastava et al. “Dropout: A Simple Way to Prevent Neural Networks from Overfitting”. In:Journal of Machine Learning Research15.56 (2014), pp. 1929–1958. url:http://jmlr.org/papers/v15/srivastava14a.

html.