Dynamic fusion of images from the visible and infrared channels of sightseeing system by complex matrix formalism
DOI:
https://doi.org/10.33577/2312-4458.25.2021.29-37Keywords:
digital image processing, image fusion, infrared imaging, Jones matrix formalism, complex vector image fusionAbstract
The employment of new mathematical and computer approaches for the fusion of target images from the visible and infrared channels of the sightseeing system (SSS) is one of the ways to increase the efficiency of the SSS of armored vehicles. Modern approaches to improving the efficiency of image fusion are aimed to increase the visibility of the target via improving the quality indices of fused images. This paper proposes a fundamentally new approach to image fusion, namely dynamic image fusion, at which the target is observed in the mode of a video clip composed of a sequence of stationary fused images obtained at different parameters of fusion, in contrast to traditional stationary image fusion, at which the decision is made from one fused image. Unlike stationary image fusion, aimed to increase the visibility of the target, the dynamic image fusion allows one to enhance the conspicuity of the target. The principle of dynamic image fusion proposed in this paper is based on matrix formalism, in which the fused image is constructed in the form of a complex vector function, which by its mathematical form is analogous to the Jones vector of elliptically polarized light wave, which in turn opens the possibility of matrix transformation of the complex vector of the fused image and consequently its parameterization by analogy with the Jones matrix formalism for the light wave. The article presents mathematical principles of matrix formalism, which is the basis for dynamic image fusion, gives examples of stationary and dynamic image fusion by the method of complex vector function and compares with the corresponding images, fused by algorithms of weight addition in the field of real and complex scalars. It is shown that by selecting weight coefficients, the general form of a complex amplitude vector image can be reduced to the algorithms of weight and averaged addition in the field of real scalars, weight amplitude and RMS-image in the field of complex scalar numbers, and geometric-mean image in the field of complex vectors, which, thereby, are partial cases of the general form of the complex amplitude image in the field of complex vectors.
The animated images obtained by the method of complex vector function illustrate the increase of conspicuity of the object of observation due to the dynamic change of the fusion parameters.
References
″Ukrayinska kompaniya archer rozrobyla vdoskonalenyj tankovyj prytsil″ [Ukrainian company Archer has developed an improved tank sight]. (2021), URL: https://mil.in.ua/uk/news/ukrayinska-kompaniya-archer-rozrobyla-vdoskonalenyj-tankovyj-prytsil/ (Accessed 28 July 2021)
Dylan Malyasov (2021), Ukraine’s Archer develops new gunner’s sight for main battle tanks. URL: https://defence-blog.com/ukraines-archer-develops-new-gunners-sight-for-main-battle-tanks/?amp&__twitter_impression=true (Accessed 28 July 2021)
Ya. Ye. Khaustov , D. Ye. Khaustov, Ye. Ryzhov, E. Lychkovskyy and Yu. A. Nastishin. (2020), Fusion of visible and infrared images via complex function. Military technical collection. № 22, pp. 20-31 DOI: https://doi.org/10.33577/2312-4458.22.2020.20-31
Ya.Ye. Khaustov, D.Ye. Khaustov, A.P. Hryvachevskyi , Ye. Ryzhov, E. Lychkovskyy , I.N. Prudyus and Yu.A. Nastishin. (2021). Complex function as a template for image fusion. Results in Optics 2 100038. https://doi.org/10.1016/j.rio.2020.100038
D.Ye. Khaustov, Yu.A. Nastishin and O.Yu. Larin (2021), ″Analiz efektyvnosti rozvidky vorozhykh tsilei na poli boiu bahatokanalnym prytsilno-sposterezhnym kompleksom″ [Analysis of the effectiveness of reconnaissance of enemy targets on the battlefield by a multi-channel sighting and surveillance system]. Military technical collection CNDI OVT.
A.H. Wertheim (2010), Visual conspicuity: A new simple standard, its reliability, validity and applicability. Ergonomics Vol. 53, No. 3, pp.421–442.
Miguel P. Eckstein (2011), Visual search: A retrospective. Journal of Vision December Vol.11, 14. doi:https://doi.org/10.1167/11.5.14
Y.M. Bowler (1990), "Towards a simplified model of visual search", in Visual Search, D. Brogan, Ed., pp. 303—309, Taylor & Francis, London, UK.
A. Toet, P. Bijl, FL. Kooi, and J.M. Valeton (1997)., Image data setfor testing search and detection models, Report TM-97-A036, TNO Human Factors Research Institute, Soesterberg, The Netherlands.
Wolfe, J. M. (2015), Visual search. In J. M. Fawcett, E. F. Risko, & A. Kingstone (Eds.), The handbook of attention (pp. 27–56). Boston Review.
Lana M. Trick and James T. Enns. (1998), LIFESPAN CHANGES IN ATTENTION: THE VISUAL SEARCH TASK Cognitive Development, 13, 369-386
William K. Pratt. (2001), Digital Image Processing: PIKS Inside, Third Edition. New York, John Wiley & Sons, Inc.
Gonzalez Rafael C. and Richard E. Woods (2002), Digital Image Processing, New Jersey, Prentice-Hall, Inc.
A.H. Wertheim (2010), Visual conspicuity: A new simple standard, its reliability, validity and applicability. Ergonomics. Vol. 53, No. 3, March 421–442.
Huckauf A., Heller D. and Nazir T.A. (1999), Lateral masking: limitations of the feature interaction account. Perception and Psychophysics, 61 (1), 177–189.
Krikke K., Wertheim A.H., and Johnson A. (2000), The role of lateral masking in visual search. Perception, 29 (Suppl), 102.
Wertheim A.H., et al. (2006), How important is lateral masking in visual search? Experimental Brain Research, 170, 387–402.
A. Broca and D. Sulzer (1902), La sensation lumineuse en function du temps, Jorr. DE Physio. Et de Pathol. Generale 4, 632–640.
Luyao Tang, Tianran Chen, Xiaolin Zhang and Muqing Liu. (2020), Experimental study on brightness perception changing effect of pulsed light in short wavelengths. Optik, Volume 217, September 164817.
Shenglong Fan, Xin Gu, Xiaolin Zhang and Muqing Liu. (2014), Human perception on pulsed red and green lights. Optical Engineering 53(6), 065105.
Khaustov D.Ye., Khaustov Ya.Ye., Ryzhov Ye., Lychkowskyy E., Vlokh R. and Nastishin Yu.A. (2021), Jones formalism for image fusion. Ukr. J. Phys. Opt. Volume 22, Issue 3, 165-180.
Azzam R.M.A. and Bashara N.M. (1977), Ellipsometry and Polarized Light. North–Holland Amsterdam, New York.
Nastyshyn S.Yu., Bolesta I.M., Tsybulia S.A., Lychkovskyy E., Yakovlev M.Yu., Ryzhov Ye., Vankevych P.I. and Nastishin Yu.A. (2018), Differential and integral Jones matrices for a cholesteric. Phys. Rev. A 97, 053804.
Nastyshyn S.Yu., Bolesta I.M., Tsybulia S.A., Lychkovskyy E., Fedorovych Z.Ya., Khaustov D.Ye., Ryzhov Ye., Vankevych P.I. and Nastishin Yu.A. (2019), Optical spatial dispersion in terms of Jones calculus. Phys. Rev. A 100. https://doi.org/10.1103/PhysRevA.100.013806. 013806.
A.F. Konstantinova, B.N. Grechushnikov, B.V. Bokut and Ye.G. Valyashko (1995), Optical Properties of Crystals. Navuka i Teknnika, Minsk.
Visual and Infrared Database image Fusion, provided for free downloading by the SMT/COPPE/Poli/UFRJ and IME–Instituto Militar de Engenharia within the CAPES/Pró–Defesa Program, in a partnership with IPqM–Instituto de Pesquisa da Marinhaat http://www02.smt.ufrj.br/~fusion/.
Ya.Ye. Khaustov, D.Ye. Khaustov, Е. Lychkovskyy, Ye.Ryzhov and Yu.A. Nastishin (2019), Image fusion for a target sightseeing system of armored vehicles. Military technical collection. № 21, pp. 28-37. DOI: https://doi.org/10.33577/2312-4458.21.2019.28-37



