Yh"o|dZddlZddlZddlZddlmZddlZddlZ ddl Z ddl Z ddl Z ddl mZGddZdS)z Created by Jaided AI Released Date: 18/08/2022 Description: DBNet text detection module. Many parts of the codes are adapted from https://github.com/MhLiao/DB N)Polygon) ConfigurableceZdZ ddZdZd Zd Zd Zd Zd d Z dZ dZ d!dZ d dZ d"dZdZ d#dZ d#dZd$dZdZdZdZ d%dZdS)&DBNetresnet18N pretrainedTcudarcp||_tjtjt dd}t |d5} tj| |_ dddn #1swxYwY| | |j ||_ ||j vr||_ nLtdd|j |||_nGtjtjt d|_|r||j |d vr?tj|j|j |d|} d } n'tj|j|} d } tj| s#t#| || ||j |d | nd|_t)j|j d |_|j d |_|j d|_dS)a DBNet text detector class Parameters ---------- backbone : str, optional Backbone to use. Options are "resnet18" and "resnet50". The default is "resnet18". weight_dir : str, optional Path to directory that contains weight files. If set to None, the path will be set to "../weights/". The default is None. weight_name : str, optional Name of the weight to use as specified in DBNet_inference.yaml or a filename in weight_dir. The default is 'pretrained'. initialize_model : Boolean, optional If True, construct the model and load weight at class initialization. Otherwise, only initial the class without constructing the model. The default is True. dynamic_import_relative_path : str, optional Relative path to 'model/detector.py'. This option is for supporting integrating this module into other modules. For example, easyocr/DBNet This should be left as None when calling this module as a standalone. The default is None. device : str, optional Device to use. Options are "cuda" and "cpu". The default is 'cuda'. verbose : int, optional Verbosity level. The default is 0. Raises ------ ValueError Raised when backbone is invalid. FileNotFoundError Raised when weight file is not found. Returns ------- None. configszDBNet_inference.yamlrNz2Invalid backbone. Current support backbone are {}.,weightsweightzUA weight with a name {} is found in DBNet_inference.yaml but cannot be find file: {}.zYA weight with a name {} is not found in DBNet_inference.yaml and cannot be find file: {}.modelBGR_MEANmin_detection_sizemax_detection_size)deviceospathjoindirname__file__openyaml safe_loadr set_relative_import_pathkeysbackbone ValueErrorformat weight_dirisfileFileNotFoundErrorinitialize_modelrnparrayrrr) selfr r# weight_namer&dynamic_import_relative_pathrverbose config_pathfid weight_path error_messages e/var/www/tools.fuzzalab.pt/emblema-extractor/venv/lib/python3.11/site-packages/easyocr/DBNet/DBNet.py__init__zDBNet.__init__sr\ gll27??8#<##..DL / / / / / / / / / / / / / / / ( 388GcddDL t|((** * *$DMMQXXY\YaYabfbnbsbsbubuYvYvwwxx x  !(DOO gll27??8+D+DiPPDO  dl84X>CCEEEE gll4?DL>+.. X' (<(<[+(V(VWWW  ! !$,x"8"A; O O O ODJj!9:: "&,/C"D"&,/C"DsBBBc f|J|tj}|D]x\}}|dkrA||d||dziLt |tr|||}xy|S)a Create relative import paths for modules specified in class. This method is recursive. Parameters ---------- configs : dict Configuration dictionary from .yaml file. dynamic_import_relative_path : str, optional Relative path to 'model/detector/'. This option is for supporting integrating this module into other modules. For example, easyocr/DBNet This should be left as None when calling this module as a standalone. The default is None. Returns ------- configs : dict Configuration dictionary with correct relative path. Nclass.) splitrsepitemsupdater isinstancedictr)r)r r+preficeskeyvalues r1rzDBNet.set_relative_import_pathjs(,777/55bf==   ICg~~SXXhS9I9I.I%J%JKLLLLeT** 99%A]^^EEc|jtd|jtj||jd|jdS)a. Load weight to model. Parameters ---------- weight_path : str Path to trained weight. Raises ------ RuntimeError Raised when the model has not yet been contructed. Returns ------- None. Nz#model has not yet been constructed.) map_locationF)strict)r RuntimeErrorload_state_dicttorchloadreval)r)r/s r1 load_weightzDBNet.load_weights`$ : DEE E ""5:k #T#T#T]b"ccc r?c|tj|jj|j|_dS)z Contruct text detection model based on the configuration in .yaml file. Parameters ---------- config : dict Configuration dictionary. Returns ------- None. N)rconstruct_class_from_config structurebuilderbuildrr)r)configs r1construct_modelzDBNet.construct_models2"=fEEOW]]^b^ijj r?c0||||t|jjtjjr@|jdkr7|jjj |j|j_dSdSdS)aO Wrapper to initialize text detection model. This model includes contructing and weight loading. Parameters ---------- model_config : dict Configuration dictionary. weight_path : str Path to trained weight. Returns ------- None. cpuN) rOrHr:rrEnn DataParallelrmoduleto)r) model_configr/s r1r&zDBNet.initialize_models \*** %%% dj&(= > > G4;RWCWCW#z/699$+FFDJ    G GCWCWr?ct|trttj|r3t j|t jd}ntd |t|tj r|d}nSt|tjjr%tj|dddddddf}nt!d|S)a Load or convert input to OpenCV BGR image numpy array. Parameters ---------- image : str, PIL.Image, or np.ndarray Image to load or convert. Raises ------ FileNotFoundError Raised when the input is a path to file (str), but the file is not found. TypeError Raised when the data type of the input is not supported. Returns ------- image : np.ndarray OpenCV BGR image. float32zCannot find {}NzYUnsupport image format. Only path-to-file, opencv BGR image, and PIL image are supported.)r:strrrr$cv2imread IMREAD_COLORastyper%r"r'ndarrayPILImageasarray TypeErrorr)images r1 get_cv2_imagezDBNet.get_cv2_images* eS ! ! yw~~e$$ H 5#*:;;BB9MM'(8(?(?(F(FGGG rz * * yLL++EE sy / / yJu%%aaaDDbDj1EEwxx x r?c|j\}}}|)t|jt|||j}||krUt t j|dz dz}t t j||z |zdz dz}nTt t j|dz dz}t t j||z |zdz dz}tj |||f}|||ffS)a Resize image such that the shorter side of the image is equal to the closest multiple of 32 to the provided detection_size. If detection_size is not provided, it will be resized to the closest multiple of 32 each side. If the original size exceeds the min-/max-detection sizes (specified in configs.yaml), it will be resized to be within the min-/max-sizes. Parameters ---------- img : np.ndarray OpenCV BGR image. detection_size : int, optional Target detection size. The default is None. Returns ------- np.ndarray Resized OpenCV BGR image. The width and height of this image should be multiple of 32. N ) shapemaxrminrintmathceilr[resize) r)imgdetection_sizeheightwidth_ new_height new_width resized_imgs r1 resize_imagezDBNet.resize_images,9q  ! !8#feTMd:e:effN E>>TY~':;;b@AAJDIj6&9E&AB&FGG"LMMIIDInr&9::R?@@ITYy5'86'AB'FGG"LMMJjy*&=>> VUO++r?ctj|ddddS)a4 Convert image array (assuming OpenCV BGR format) to image tensor. Parameters ---------- image : np.ndarray OpenCV BGR image. Returns ------- torch.tensor Tensor image with 4 dimension [batch, channel, width, height]. rr)rE from_numpypermutefloat unsqueezerds r1image_array2tensorzDBNet.image_array2tensor sA&&..q!Q77==??II!LLLr?c||jz dz S)z Normalize image by substracting BGR mean and divided by 255 Parameters ---------- image : np.ndarray OpenCV BGR image. Returns ------- np.ndarray OpenCV BGR image. go@)rrds r1normalize_imagezDBNet.normalize_images %u,,r?c||}|||\}}||}||}||fS)a$ Wrapper to load and convert an image to an image tensor Parameters ---------- image : path-to-file, PIL.Image, or np.ndarray Image to load or convert. detection_size : int, optional Target detection size. The default is None. Returns ------- img : torch.tensor Tensor image with 4 dimension [batch, channel, width, height].. original_shape : tuple A tuple (height, width) of the original input image before resizing. rq)rfrxrr)r) image_pathrqrporiginal_shapes r1 load_imagezDBNet.load_image,se$   + +"//n/UU^""3''%%c**N""r?chtfd|D\}}tj|d|fS)a Wrapper to load or convert list of multiple images to a single image tensor. Multiple images are concatenated together on the first dimension. Parameters ---------- images : a list of path-to-file, PIL.Image, or np.ndarray Image to load or convert. detection_size : int, optional Target detection size. The default is None. Returns ------- img : torch.tensor A single tensor image with 4 dimension [batch, channel, width, height]. original_shape : tuple A list of tuples (height, width) of the original input image before resizing. c>g|]}|S)r)r).0rerqr)s r1 z%DBNet.load_images..Xs:(=(=(=,1)-P^(_(_(=(=(=r?r)dim)ziprEcat)r)imagesrqoriginal_shapess` ` r1 load_imageszDBNet.load_imagesEsZ&#&(=(=(=(=(=5;(=(=(=#>yq)))?::r?皙?Fc |||} g} g} t|dD]} || \} }|r,||| | | || |||\}}n+||| | | || |||\}}| || |t dt | | D\} } | | fS)a Translate probability heatmap tensor to text region boudning boxes. Parameters ---------- image_tensor : torch.tensor Image tensor. original_shapes : tuple Original size of the image (height, width) of the input image (before rounded to the closest multiple of 32). hmap : torch.tensor Probability heatmap tensor. text_threshold : float, optional Minimum probability for each pixel of heatmap tensor to be considered as a valid text pixel. The default is 0.2. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. as_polygon : boolean, optional If True, return the bounding box as polygon (fine vertrices), otherwise return as rectangular. The default is False. Returns ------- boxes_batch : list of lists Bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. ) thresholdr)bbox_min_score bbox_min_sizemax_candidatesc g|]=\}}t|dkr!tdt||Dnddg>S)rc(g|]\}}|dk ||fSr)rboxscores r1rz.DBNet.hmap2bbox...s90d0d0d4?SY^abYbYb25e YbYbYbr?r)anyr)rboxesscoress r1rz#DBNet.hmap2bbox..s*`*`*`.=eV36fqj//+O#0d0dCFufCUCU0d0d0d+/+/HJ2w*`*`*`r?)binarizerangesizepolygons_from_bitmapboxes_from_bitmapappendr)r) image_tensorrhmaptext_thresholdrrr as_polygon segmentation boxes_batch scores_batch batch_indexrrrsrrs r1 hmap2bboxzDBNet.hmap2bbox\s[\}}T~}FF    !2!21!5!566 ( (K+K8MFE I $ 9 9(,[(9(4[(A(-(.9G8E9G!:!I!I vv!% 6 6(,[(9(4[(A(-(.9G8E9G!7!I!I v   u % % %    ' ' ' '$'*`*`ADKQ]@^@^*`*`*`%)! \ L((r?c||kS)a Apply threshold to return boolean tensor. Parameters ---------- tensor : torch.tensor input tensor. threshold : float Threshold. Returns ------- torch.tensor Boolean tensor. r)r)tensorrs r1rzDBNet.binarizes" !!r?c|ddksJ|d}|d}|j\} } g} g} t j|dztj t j t j \} }|dkr | d|} | D] }dt j |dz}t j ||d}|d}|jddkrX|||d d }||kr|jdd kr,||d }t#|dkrn|d d }||d \}}||d zkrt'|t(s(|}|}tjtj|dddf| z |zd||dddf<tjtj|dddf| z |zd||dddf<| || | | | fS)a Translate boolean tensor to fine polygon indicating text bounding boxes Parameters ---------- hmap : torch.tensor Probability heatmap tensor. segmentation : torch.tensor Segmentataion tensor. dest_width : TYPE target width of the output. dest_height : TYPE target width of the output. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. Returns ------- boxes_batch : list of lists Polygon bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. rrNgMb`?T)rYrzrYrzg@) unclip_ratio)rYrrz)rrQnumpydetachrir[ findContoursr^r'uint8 RETR_LISTCHAIN_APPROX_SIMPLE arcLength approxPolyDPreshapebox_score_fastuncliplenget_mini_boxesr:rlitemcliproundrtolist)r)rr dest_width dest_heightrrrbitmaprrrsrrcontoursrtcontourepsilonapproxpointsrrssides r1rzDBNet.polygons_from_bitmapsP  ##q((((!!##))++A.xxzz  ""((**1- & CZ   ) ) M3244 ! A  0H! !! !GcmGT:::G%gw==F^^G,,F|A""''fnnR.C.CDDE~%%|A""kk&sk;;s88a<< ++b!$$C**3;;z+B+BCCHAu}q(((j#.. 1'__.. )..00 QQQTU*Z788!ZIIC1IQQQTV+k9::A{LLC1I LL & & & MM% f}r?cf|ddksJ|d}|d}|j\} } t j|dztj t j t j \} } |dkrtt| |} nt| } tj| ddftj}tj| ftj}t#| D]}| |}||\}}||kr*tj|}|||dd}||kro||ddd}||\}}||dzkrtj|}t/|t0s(|}|}tjtj|dddf| z |zd||dddf<tjtj|dddf| z |zd||dddf<|tj||ddddf<|||<||fS) a Translate boolean tensor to fine polygon indicating text bounding boxes Parameters ---------- hmap : torch.tensor Probability heatmap tensor. segmentation : torch.tensor Segmentataion tensor. dest_width : TYPE target width of the output. dest_height : TYPE target width of the output. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. Returns ------- boxes_batch : list of lists Polygon bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. rrrrrzdtyperYN)rrQrrrir[rr^r'rrrrkrzerosint16rXrrr(rrrr:rlrrrr)r)rrrrrrrrrrrsrrt num_contoursrrindexrrrrrs r1rzDBNet.boxes_from_bitmapsN  ##q((((!!##))++A.xxzz  ""((**1- &#CZ//99M3+BDD ! A  s8}}n==LLx==L,1-RX>>>z&DBNet.get_mini_boxes..zs 1r?)r=)rrrzrrrrrz)r[ minAreaRectsortedlist boxPointsrk) r)r bounding_boxrindex_1index_2index_3index_4rs r1rzDBNet.get_mini_boxesxsw// S]<8899~~NNN-7*'7 !9Q<&)A, & &GGGGG !9Q<&)A, & &GGGGGgwgw1C Q((((r?c |jdd\}}|}tjtj|dddftjd|dz }tjtj|dddf tjd|dz }tjtj|dddftjd|dz }tjtj|dddf tjd|dz } tj | |z dz||z dzftj } |dddf|z |dddf<|dddf|z |dddf<tj | |dddtjdtj||| dz||dzf| dS)a- Calculate total score of each bounding box Parameters ---------- hmap : torch.tensor Probability heatmap tensor. box_ : list Rectanguar bounding box. Returns ------- float Confidence score. NrzrrrrY)ricopyr'rfloorrkr^int32rnrjrrr[fillPolyrmean) r)rbox_hwrxminxmaxyminymaxmasks r1rzDBNet.box_score_fasts  z"1"~1iikkwrxAAAqD 0077AA1a!eLLwrws111a4y}}//66rx@@!QUKKwrxAAAqD 0077AA1a!eLLwrws111a4y}}//66rx@@!QUKKxq$+/:"(KKK1I$AAAqD 1I$AAAqD  T3;;q"a0077AA1EEExT$q&[$tAv+56==a@@r?c:|j|dS)ap Run the model to obtain a heatmap tensor from a image tensor. The heatmap tensor indicates the probability of each pixel being a part of text area. Parameters ---------- image_tensor : torch.tensor Image tensor. Returns ------- torch.tensor Probability heatmap tensor. F)training)rforward)r)rs r1 image2hmapzDBNet.image2hmapsz!!,!???r?c 6t|ts|g}|||\} } tj5|| } || | | |||||\} } dddn #1swxYwY|r| | fS| S)az Wrapper to run the model on an input image to get text bounding boxes. Parameters ---------- image : path-to-file, PIL.Image, or np.ndarray Image to load or convert. text_threshold : float, optional Minimum probability for each pixel of heatmap tensor to be considered as a valid text pixel. The default is 0.2. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. detection_size : int, optional Target detection size. Please see docstring under method resize_image() for explanation. The default is None. as_polygon : boolean, optional If true, return the bounding boxes as find polygons, otherwise, return as rectagular. The default is False. return_scores : boolean, optional If true, return confidence score along with the text bounding boxes. The default is False. Returns ------- list of lists Text bounding boxes. If return_scores is set to true, another list of lists will also be returned. r)rrrrrN)r:rrrEno_gradrr)r)rerrrrrqr return_scoresrrr batch_boxes batch_scoress r1 inferencezDBNet.inferences\%&& GE(,(8(8Q_(8(`(`% o ]__ N N??<00D(,|7F7;HVHVGTHVBL)7)N)N %K N N N N N N N N N N N N N N N   , , s6BB B )rNr TNr rrr)rrrrF)rrr)r)rrrrNFF)__name__ __module__ __qualname__r2rrHrOr&rfrxrrrrrrrrrrrrr rr?r1rrs&"+$(04 SESESESEj@. k k kGGG*!!!F",",",",HMMM --- ####2;;;;6$'#&"##$"O)O)O)O)b"""0/2-../YYYY@,/*++,R&R&R&R&h))).AAA<@@@&$'#&"##$#'$"'@@@@@@r?r)__doc__rrmrshapely.geometryr PIL.Imager`rr'r[rrEmodel.constructorrrrr?r1rs $$$$$$  ++++++j j j j j j j j j j r?