Contextual word embeddings ( e.g. GPT , BERT , ELMo , etc . ) have demonstrated state-of-the-art performance on various NLP tasks . Recent work with the multilingual version of BERT has shown that the model performs very well in cross-lingual settings , even when only labeled English data is used to finetune the model . We improve upon multilingual BERT 's zero-resource cross-lingual performance via adversarial learning . A natural question is how such networks represent this information internally . This paper describes qualitative and quantitative investigations of one particularly effective model , BERT . At a high level , linguistic features seem to be represented in separate semantic and syntactic subspaces . BERT is a recent language representation model that has surprisingly performed well in diverse language understanding benchmarks . This result indicates the possibility that BERT networks capture structural information about language . In this work , we provide novel support for this claim by performing a series of experiments to unpack the elements of English language structure learned by BERT . Multilingual BERT ( mBERT ) trained on 104 languages has shown surprisingly good cross-lingual performance on several NLP tasks , even without explicit cross-lingual signals . However , these evaluations have focused on cross-lingual transfer with high-resource languages , covering only a third of the languages covered by mBERT . We aim to identify architectural properties of BERT as well as linguistic properties of lan- guages that are necessary for BERT to become multilingual . To allow for fast experimenta- tion we propose an efficient setup with small BERT models and synthetic as well as natu- ral data . Overall , we identify six elements that are potentially necessary for BERT to be mul- tilingual . For instance , the methods suffer from data-distribution differences and tremendous manual efforts spent on data labeling . Therefore , a monitoring and early warning method for new cyber-telecom crime platforms based on the BERT migration learning model is proposed . This method first identifies the text data and their tags , and then performs migration training based on a pre-training model . To this end , we generalize the standard BERT model to a multi-task learning setting where we couple BERT 's masked language modeling and next sentence prediction objectives with an auxiliary task of binary word relation classification . Our experiments suggest that our " Lexically Informed " BERT ( LIBERT ) , specialized for the word-level semantic similarity , yields better performance than the lexically blind " vanilla " BERT on several language understanding tasks . Concretely , LIBERT outperforms BERT in 9 out of 10 tasks of the GLUE benchmark and is on a par with BERT in the remaining one . In addition , LSTM and Gradient Boosted Tree models have been built to perform the task and comparative results are provided for all three models . Fine-tuned BERT model could achieve an accuracy of 97.021 % on NewsFN data and is able to outperform the other two models by approximately eight percent . We propose a novel discriminative re-ranker to improve the performance of generative text-to-SQL models by extracting the best SQL query from the beam output predicted by the text-to-SQL generator , resulting in improved performance in the cases where the best query was in the candidate list , but not at the top of the list . We build the re-ranker as a schema agnostic BERT fine-tuned classifier . We analyze relative strengths of the text-to-SQL and re-ranker models across different query hardness levels , and suggest how to combine the two models for optimal performance . In this paper , we propose a novel Learning-to-Rescore ( L2RS ) mechanism , which is specialized for utilizing a wide range of textual information from the state-of-the-art NLP models and automatically deciding their weights to rescore the N-best lists for ASR systems . Specifically , we incorporate features including BERT sentence embedding , topic vector , and perplexity scores produced by n-gram LM , topic modeling LM , BERT LM and RNNLM to train a rescoring model . We conduct extensive experiments based on a public dataset , and experimental results show that L2RS outperforms not only traditional rescoring methods but also its deep neural network counterparts by a substantial improvement of 20.67 % in terms of NDCG@10 . By multi-tasking on such a broad large-scale set of data , we hope to both move towards and measure progress in producing a single unified agent that can perceive , reason and converse with humans in an open-domain setting . We show that such multi-tasking improves over a BERT pre-trained baseline , largely due to multi-tasking with very large dialogue datasets in a similar domain , and that the multi-tasking in general provides gains to both text and image-based tasks using several metrics in both the fine-tune and task transfer settings . We obtain state-of-the-art results on many of the tasks , providing a strong baseline for this challenge . MT - DNN not only leverages large amounts of cross - task data , but also benefits from a regularization effect that leads to more general representations to help adapt to new tasks and domains . MT - DNN extends the model proposed in liu2015mtl by incorporating a pre - trained bidirectional transformer language model , known as BERT bert2018 . MT - DNN obtains new state - of - the - art results on ten NLU tasks , including SNLI , SciTail , and eight out of nine GLUE tasks , pushing the GLUE benchmark to 82.2 % ( 1.8 % absolute improvement ) . As shown in Figure 1 , the lower layers ( i.e. , text encoding layers ) are shared across all tasks , while the top layers are task - specific , combining different types of NLU tasks such as single - sentence classification , pairwise text classification , text similarity , and relevance ranking . Similar to the BERT model , MT - DNN is trained in two stages : pre - training and fine - tuning . Unlike BERT , MT - DNN uses MTL in the fine - tuning stage with multiple task - specific layers in its model architecture . A classifier can then be learned on top of the resulting encoder , which is commonly further fine - tuned during this supervised training . Despite the strong performance of these approaches in monolingual settings , we argue that fixed - length approaches provide a more generic , flexible and compatible representation form for our multilingual scenario , and our model indeed outperforms the multilingual BERT model Devlin:2018:arxiv_bert in zero - shot transfer ( see Section [ reference ] ) . Our transfer results are strong and homogeneous across all languages : for 11 of them , the zero - short performance is ( at most ) 5 % lower than the one on English , including distant languages like Arabic , Chinese and Vietnamese , and we also achieve remarkable good results on low - resource languages like Swahili . In contrast , BERT achieves excellent results on English , outperforming our system by 7.5 points , but its zero - shot cross - lingual transfer performance is much weaker . For instance , the loss in accuracy for both Arabic and Chinese is 2.5 points for our system , compared to 19.3 and 17.6 points for BERT . In this paper , we examine the performance of classifiers trained using the media frames Corpus ( MFC ) ( Card et al. , 2015 ) ; a collection of US news labelled with fifteen different frame categories . Specifically , we compare pre-trained language models ( XLNet , Bert , and Roberta ) , fine-tuned using MFC , against results from the literature and simpler models in their ability to predict frames from text . We also test these models on a new corpus that we have derived from Australian parliamentary speeches . To address this issue , we explore a number of memory-light model reduction strategies that do not require model pre-training from scratch . The experimental results show that we are able to prune BERT , RoBERTa and XLNet models by up to 40 % , while maintaining up to 98 % of their original performance . We also show that our pruned models are on par with DistilBERT in terms of both model size and performance . We propose an approach to automatically create a training dataset for this problem . We utilize recently proposed XLNet and BERT architectures for this problem and find that they provide much better performance than the Siamese model , previously applied for this problem . Our model gives a good performance even when trained on one vertical and tested across different verticals . Second , we develop a probe to measure how much can a model cheat ( via non-multifact reasoning ) on existing tests and our sufficiency test . Third , we conduct experiments using a transformer based model ( XLNet ) , demonstrating that the sufficiency transform not only reduces the amount of non-multifact reasoning in this model by 6.5 % but is also harder to cheat -- a non-multifact model sees a 20.8 % ( absolute ) reduction in score compared to previous metrics . Clinical notes contain rich data , which is unexploited in predictive modeling compared to structured data . In this work , we developed a new text representation Clinical XLNet for clinical notes which also leverages the temporal information of the sequence of the notes . We evaluated our models on prolonged mechanical ventilation prediction problem and our experiments demonstrated that Clinical XLNet outperforms the best baselines consistently . In the era of big data , mining the emotional tendency of opinions through natural language processing technology is meaningful for the timely understanding of human behavior data . Nowadays , Generalized Autoregressive Pre-training Language Modeling ( XLNet ) can not only capture bidirectional contextual knowledge but also learn the word dependency . However , its sentence-level representation did n't take broad features into account . The generators are simulated with a Simplified Sync hronous Machine block . Universal transformer blocks ( two-windings and three-windings ) are used to model the two trans formers . Saturation is implemented on the transform er connected at bus B2 . The expressions and their targets were analyzed as a character-level sequence transduction task in which the encoder and decoder are built on attention mechanisms . Three models were trained to understand and evaluate symbolic variables and expressions in mathematics : ( 1 ) the self-attentive and feed-forward transformer without recurrence or convolution , ( 2 ) the universal transformer with recurrence , and ( 3 ) the adaptive universal transformer with recurrence and adaptive computation time . The models respectively achieved test accuracies as high as 76.1 % , 78.8 % and 84.9 % in evaluating the expressions to match the target values . The invention relates to an electricity-stealing-preventing electric power system , and belongs to the field of electric power systems . The electricity-stealing-preventing electric power system comprises a generating station , a power transmission grid , a power distribution transformer , a power distribution grid , a user transformer and an anti-theft electricity meter ; the generating station is connected to the power transmission grid for supplying power to the power transmission grid ; a primary side of the power distribution transformer is connected to the power transmission grid while a secondary side of the power distribution transformer is connected to the power distribution grid ; a primary side of the user transformer is connected with the power distribution grid while a secondary side of the user transformer is connected with the anti-theft electricity meter ; the anti-theft electricity meter comprises an electricity meter body , an anti-theft transformer and an anti-theft shell ; the electricity meter body and the anti-theft transformer are in series connection and the serially connected electricity meter body and the anti-theft transformer are installed in the anti-theft shell ; the output end of the anti-theft electricity meter supplies power for loads ; and the input voltage of the anti-theft transformer is different from the working voltage of the loads . The electricity-stealing-preventing electric power system provides security assurance for the load terminals ; the electricity-stealing phenomenon that domestic appliances or other loads are directly connected to the input end of the electricity meter by some people can be prevented ; the voltage before entering the electricity meter is the voltage that can not be used directly for domestic appliances ; and the anti-theft electricity meter can be directly used for the loads after the voltage is converted by the anti-theft electricity meter . Audio-Visual Scene-Aware Dialog ( AVSD ) is a task to generate responses when chatting about a given video , which is organized as a track of the 8th Dialog System Technology Challenge ( DSTC8 ) . To solve the task , we propose a universal multimodal transformer and introduce the multi-task learning method to learn joint representations among different modalities as well as generate informative and fluent responses . Our method extends the natural language generation pre-trained model to multimodal dialogue generation task . And because of its sequential nature , it is less effective at learning global dependencies among amino acids than existing transformer architectures . We propose the Universal Transforming Geometric Network ( UTGN ) , an end-to-end differentiable model that uses the encoder portion of the Universal Transformer architecture as an alternative for internal representations . Our experiments show that compared to RGN , UTGN achieve a $ 1.7 $ \si{\angstrom } improvement on the free modeling portion and a $ 0.7 $ \si{\angstrom } improvement on the template based modeling of the CASP12 competition . This technological invention pull distribution combined transformer that holds concurrently mainly used urban rail transit , tram , trolley bus and factories and miness pull step -down power supply system , set up traction transformer with the distribution transformer unification . This technological invention pull distribution combined transformer that holds concurrently has concurrently and draws power supply and step -down distribution function , and the high -pressure side is every to set up a coil mutually , and the low tension side sets up respectively and draws supply power coil and low -voltage distribution coil . This paper describes a language representation model which combines the Bidirectional Encoder Representations from Transformers ( BERT ) learning mechanism described in Devlin et al. ( 2018 ) with a generalization of the Universal Transformer model described in Dehghani et al. ( 2018 ) . We further improve this model by adding a latent variable that represents the persona and topics of interests of the writer for each training example . In this study , we investigate the use of electromagnetic coupling for unifying the transformers used for the simultaneous transmission of energy and information . We studied the transmission of energy and information by wearing a unified transcutaneous transformer . In this system , it is important to minimize the electromagnetic coupling between the energy and information transmissions . The invention discloses a method for compensating a dead area of a universal frequency transformer . The method comprises the following steps of : ( 1 ) acquiring three-phase current signals IU , IV and IW of a motor , and transmitting the signals to a digital signal processor to perform digital filtration by adopting a hardware filter ; and ( 2 ) performing linear processing on the compensation magnitude according to the magnitude and positive and negative of the detected current , namely when the current of one phase is positive , correspondingly increasing a dead area time to the switch-on time of an upper bridge arm Qx of an inverting bridge corresponding to the current , and correspondingly reducing a dead area time from the switch-on time of a lower bridge arm Qy of the inverting bridge corresponding to the current ; and when the current of the phase is negative , correspondingly reducing a dead area time from the switch-on time of the upper bridge arm Qx of the inverting bridge corresponding to the current , and correspondingly increasing a dead area time to the switch-on time of the lower bridge arm Qy of the inverting bridge corresponding to the current . Some researchers try to enhance models ’ language comprehension ability by employing the pre-trained language models , but they neglect the importance of external knowledge in specific tasks . In this paper , we propose a novel universal transformer-based architecture for dialogue system , the Multiple Knowledge Syncretic Transformer ( MKST ) , which fuses multi-knowledge in open-domain conversation . Firstly , the model is pre-trained on a large-scale corpus to learn commonsense knowledge . Through the experiment the proposed approach has been proved to be better in both semantic meaning and quality than the existing popular algorithms of keyword extraction . In addition , we propose to use contextual features from bidirectional transformers to automatically label short-sentence corpus with keywords and keyphrases to build the ground truth . This process avoids the human time to label the keywords and do not need any prior knowledge . Both design methods are applied to two sample high voltage transformers . The measured performances of the as-built transformers highlight the limitations in using the conventional method , which are overcome using the reverse approach . The auxiliary isolating switch comprises a supporting frame , a motor , a lead screw , a static contact , a movable contact component , an auxiliary switch , a cam , a first relay JD1 and a second relay JD2 , the motor is positioned on the supporting frame , the lead screw is supported on the supporting frame and driven by the motor , the static contact is positioned on the supporting frame , the movable contact component is composed of a movable contact support and a movable contact fixed on the movable contact support , the auxiliary switch is provided with normally-open and normally-closed contact switches K1 and K2 , the cam is supported on the supporting frame and used for driving the auxiliary switch to act , and the first relay JD1 and the second relay JD2 are arranged on the supporting frame . The auxiliary isolating switch is simple , reasonable and unique in structure , safe and reliable in running , low in cost and especially suitable for CRH5-type motor train unit traction auxiliary current transformers and is used for electric isolating of the auxiliary current transformers . For grid optimization , DSOs have implemented dynamic voltage control in substations , adapted the grid structure for renewables , applied the “ N minus zero ” rule for planning grid operation and used the capability of renewable energy systems to provide reactive power . For grid expansion , DSOs have installed voltage regulators , voltage-regulated distribution transformers , substations used exclusively for renewables , and express feeders that connect substations with the generation centers . This practical experience should also prove relevant for other countries currently planning the expansion of distributed renewable energy systems . This paper presents a control structure of Solid-State Transformers for three-phase AC/AC , to reduce the required size of the capacitors with load unbalanced compensation capability . The structure discussed in this paper consists of an AC/DC converter based on cascaded H-bridge ( CHB ) converters , isolated DC/DC converters , and a DC/AC inverter . Smart-grid-enabled transformers have unique challenges in rural settings . One such challenge is that of effective and reliable communication of data . However , in the case of a magnetic component , it is desirable to minimize the use because of the reduction of the traveling distance due to the increase in weight and volume and the insulation breakdown between the transformer windings due to vibration . Therefore , this paper proposes a dc-link voltage-variable integrated system that uses OBC and LDC transformers in common . The proposed circuit is divided into four operation modes . The device has the size of the structure of a panel of the electrical accessory , and is mounted by using an electrical accessory mounting box . The circuit structure of the device comprises : a network processor , for controlling and processing the network application function of the network access device ; a wireless receiving and sending circuit , connected to the network processor and used for amplification , sending , and receiving of a wireless signal ; multiple network isolation transformers , provided between a wired network interface and the network processor of the network access device ; a power supply module ; and multiple toggle switches for setting the working mode . The network access device is encapsulated into a socket form of the electrical accessory , and is mounted at the mounting position of the electrical accessory of building facilities . With the application of sensor networks in industrial , commercial , and medical areas , sensor operating system should be revisited to handle dependability problem . In this paper , we present a novel component-based sensor operating system , MTOS ( Micro-Transformers Operating System ) , which supports adaptive dynamic reconfiguration for resource-constrained sensor nodes to achieve dependability . Through error-driven reconfiguration and change-driven reconfiguration , MTOS can push a system into a stable state in which errors can be recovered with minimized side effects and changes can be safely made without interruption of normal services .
Contextual word embeddings ( e.g. GPT , BERT , ELMo , etc . ) have demonstrated state-of-the-art performance on various NLP tasks . Recent work with the multilingual version of BERT has shown that the model performs very well in cross-lingual settings , even when only labeled English data is used to finetune the model . We improve upon multilingual BERT 's zero-resource cross-lingual performance via adversarial learning .
A natural question is how such networks represent this information internally . This paper describes qualitative and quantitative investigations of one particularly effective model , BERT . At a high level , linguistic features seem to be represented in separate semantic and syntactic subspaces .
BERT is a recent language representation model that has surprisingly performed well in diverse language understanding benchmarks . This result indicates the possibility that BERT networks capture structural information about language . In this work , we provide novel support for this claim by performing a series of experiments to unpack the elements of English language structure learned by BERT .
Multilingual BERT ( mBERT ) trained on 104 languages has shown surprisingly good cross-lingual performance on several NLP tasks , even without explicit cross-lingual signals . However , these evaluations have focused on cross-lingual transfer with high-resource languages , covering only a third of the languages covered by mBERT .
We aim to identify architectural properties of BERT as well as linguistic properties of lan- guages that are necessary for BERT to become multilingual . To allow for fast experimenta- tion we propose an efficient setup with small BERT models and synthetic as well as natu- ral data . Overall , we identify six elements that are potentially necessary for BERT to be mul- tilingual .
For instance , the methods suffer from data-distribution differences and tremendous manual efforts spent on data labeling . Therefore , a monitoring and early warning method for new cyber-telecom crime platforms based on the BERT migration learning model is proposed . This method first identifies the text data and their tags , and then performs migration training based on a pre-training model .
To this end , we generalize the standard BERT model to a multi-task learning setting where we couple BERT 's masked language modeling and next sentence prediction objectives with an auxiliary task of binary word relation classification . Our experiments suggest that our " Lexically Informed " BERT ( LIBERT ) , specialized for the word-level semantic similarity , yields better performance than the lexically blind " vanilla " BERT on several language understanding tasks . Concretely , LIBERT outperforms BERT in 9 out of 10 tasks of the GLUE benchmark and is on a par with BERT in the remaining one .
In addition , LSTM and Gradient Boosted Tree models have been built to perform the task and comparative results are provided for all three models . Fine-tuned BERT model could achieve an accuracy of 97.021 % on NewsFN data and is able to outperform the other two models by approximately eight percent .
We propose a novel discriminative re-ranker to improve the performance of generative text-to-SQL models by extracting the best SQL query from the beam output predicted by the text-to-SQL generator , resulting in improved performance in the cases where the best query was in the candidate list , but not at the top of the list . We build the re-ranker as a schema agnostic BERT fine-tuned classifier . We analyze relative strengths of the text-to-SQL and re-ranker models across different query hardness levels , and suggest how to combine the two models for optimal performance .
In this paper , we propose a novel Learning-to-Rescore ( L2RS ) mechanism , which is specialized for utilizing a wide range of textual information from the state-of-the-art NLP models and automatically deciding their weights to rescore the N-best lists for ASR systems . Specifically , we incorporate features including BERT sentence embedding , topic vector , and perplexity scores produced by n-gram LM , topic modeling LM , BERT LM and RNNLM to train a rescoring model . We conduct extensive experiments based on a public dataset , and experimental results show that L2RS outperforms not only traditional rescoring methods but also its deep neural network counterparts by a substantial improvement of 20.67 % in terms of NDCG@10 .
By multi-tasking on such a broad large-scale set of data , we hope to both move towards and measure progress in producing a single unified agent that can perceive , reason and converse with humans in an open-domain setting . We show that such multi-tasking improves over a BERT pre-trained baseline , largely due to multi-tasking with very large dialogue datasets in a similar domain , and that the multi-tasking in general provides gains to both text and image-based tasks using several metrics in both the fine-tune and task transfer settings . We obtain state-of-the-art results on many of the tasks , providing a strong baseline for this challenge .
MT - DNN not only leverages large amounts of cross - task data , but also benefits from a regularization effect that leads to more general representations to help adapt to new tasks and domains . MT - DNN extends the model proposed in liu2015mtl by incorporating a pre - trained bidirectional transformer language model , known as BERT bert2018 . MT - DNN obtains new state - of - the - art results on ten NLU tasks , including SNLI , SciTail , and eight out of nine GLUE tasks , pushing the GLUE benchmark to 82.2 % ( 1.8 % absolute improvement ) .
As shown in Figure 1 , the lower layers ( i.e. , text encoding layers ) are shared across all tasks , while the top layers are task - specific , combining different types of NLU tasks such as single - sentence classification , pairwise text classification , text similarity , and relevance ranking . Similar to the BERT model , MT - DNN is trained in two stages : pre - training and fine - tuning . Unlike BERT , MT - DNN uses MTL in the fine - tuning stage with multiple task - specific layers in its model architecture .
A classifier can then be learned on top of the resulting encoder , which is commonly further fine - tuned during this supervised training . Despite the strong performance of these approaches in monolingual settings , we argue that fixed - length approaches provide a more generic , flexible and compatible representation form for our multilingual scenario , and our model indeed outperforms the multilingual BERT model Devlin:2018:arxiv_bert in zero - shot transfer ( see Section [ reference ] ) .
Our transfer results are strong and homogeneous across all languages : for 11 of them , the zero - short performance is ( at most ) 5 % lower than the one on English , including distant languages like Arabic , Chinese and Vietnamese , and we also achieve remarkable good results on low - resource languages like Swahili . In contrast , BERT achieves excellent results on English , outperforming our system by 7.5 points , but its zero - shot cross - lingual transfer performance is much weaker . For instance , the loss in accuracy for both Arabic and Chinese is 2.5 points for our system , compared to 19.3 and 17.6 points for BERT .
In this paper , we examine the performance of classifiers trained using the media frames Corpus ( MFC ) ( Card et al. , 2015 ) ; a collection of US news labelled with fifteen different frame categories . Specifically , we compare pre-trained language models ( XLNet , Bert , and Roberta ) , fine-tuned using MFC , against results from the literature and simpler models in their ability to predict frames from text . We also test these models on a new corpus that we have derived from Australian parliamentary speeches .
To address this issue , we explore a number of memory-light model reduction strategies that do not require model pre-training from scratch . The experimental results show that we are able to prune BERT , RoBERTa and XLNet models by up to 40 % , while maintaining up to 98 % of their original performance . We also show that our pruned models are on par with DistilBERT in terms of both model size and performance .
We propose an approach to automatically create a training dataset for this problem . We utilize recently proposed XLNet and BERT architectures for this problem and find that they provide much better performance than the Siamese model , previously applied for this problem . Our model gives a good performance even when trained on one vertical and tested across different verticals .
Second , we develop a probe to measure how much can a model cheat ( via non-multifact reasoning ) on existing tests and our sufficiency test . Third , we conduct experiments using a transformer based model ( XLNet ) , demonstrating that the sufficiency transform not only reduces the amount of non-multifact reasoning in this model by 6.5 % but is also harder to cheat -- a non-multifact model sees a 20.8 % ( absolute ) reduction in score compared to previous metrics .
Clinical notes contain rich data , which is unexploited in predictive modeling compared to structured data . In this work , we developed a new text representation Clinical XLNet for clinical notes which also leverages the temporal information of the sequence of the notes . We evaluated our models on prolonged mechanical ventilation prediction problem and our experiments demonstrated that Clinical XLNet outperforms the best baselines consistently .
In the era of big data , mining the emotional tendency of opinions through natural language processing technology is meaningful for the timely understanding of human behavior data . Nowadays , Generalized Autoregressive Pre-training Language Modeling ( XLNet ) can not only capture bidirectional contextual knowledge but also learn the word dependency . However , its sentence-level representation did n't take broad features into account .
The generators are simulated with a Simplified Sync hronous Machine block . Universal transformer blocks ( two-windings and three-windings ) are used to model the two trans formers . Saturation is implemented on the transform er connected at bus B2 .
The expressions and their targets were analyzed as a character-level sequence transduction task in which the encoder and decoder are built on attention mechanisms . Three models were trained to understand and evaluate symbolic variables and expressions in mathematics : ( 1 ) the self-attentive and feed-forward transformer without recurrence or convolution , ( 2 ) the universal transformer with recurrence , and ( 3 ) the adaptive universal transformer with recurrence and adaptive computation time . The models respectively achieved test accuracies as high as 76.1 % , 78.8 % and 84.9 % in evaluating the expressions to match the target values .
The invention relates to an electricity-stealing-preventing electric power system , and belongs to the field of electric power systems . The electricity-stealing-preventing electric power system comprises a generating station , a power transmission grid , a power distribution transformer , a power distribution grid , a user transformer and an anti-theft electricity meter ; the generating station is connected to the power transmission grid for supplying power to the power transmission grid ; a primary side of the power distribution transformer is connected to the power transmission grid while a secondary side of the power distribution transformer is connected to the power distribution grid ; a primary side of the user transformer is connected with the power distribution grid while a secondary side of the user transformer is connected with the anti-theft electricity meter ; the anti-theft electricity meter comprises an electricity meter body , an anti-theft transformer and an anti-theft shell ; the electricity meter body and the anti-theft transformer are in series connection and the serially connected electricity meter body and the anti-theft transformer are installed in the anti-theft shell ; the output end of the anti-theft electricity meter supplies power for loads ; and the input voltage of the anti-theft transformer is different from the working voltage of the loads . The electricity-stealing-preventing electric power system provides security assurance for the load terminals ; the electricity-stealing phenomenon that domestic appliances or other loads are directly connected to the input end of the electricity meter by some people can be prevented ; the voltage before entering the electricity meter is the voltage that can not be used directly for domestic appliances ; and the anti-theft electricity meter can be directly used for the loads after the voltage is converted by the anti-theft electricity meter .
Audio-Visual Scene-Aware Dialog ( AVSD ) is a task to generate responses when chatting about a given video , which is organized as a track of the 8th Dialog System Technology Challenge ( DSTC8 ) . To solve the task , we propose a universal multimodal transformer and introduce the multi-task learning method to learn joint representations among different modalities as well as generate informative and fluent responses . Our method extends the natural language generation pre-trained model to multimodal dialogue generation task .
And because of its sequential nature , it is less effective at learning global dependencies among amino acids than existing transformer architectures . We propose the Universal Transforming Geometric Network ( UTGN ) , an end-to-end differentiable model that uses the encoder portion of the Universal Transformer architecture as an alternative for internal representations . Our experiments show that compared to RGN , UTGN achieve a $ 1.7 $ \si{\angstrom } improvement on the free modeling portion and a $ 0.7 $ \si{\angstrom } improvement on the template based modeling of the CASP12 competition .
This technological invention pull distribution combined transformer that holds concurrently mainly used urban rail transit , tram , trolley bus and factories and miness pull step -down power supply system , set up traction transformer with the distribution transformer unification . This technological invention pull distribution combined transformer that holds concurrently has concurrently and draws power supply and step -down distribution function , and the high -pressure side is every to set up a coil mutually , and the low tension side sets up respectively and draws supply power coil and low -voltage distribution coil .
This paper describes a language representation model which combines the Bidirectional Encoder Representations from Transformers ( BERT ) learning mechanism described in Devlin et al. ( 2018 ) with a generalization of the Universal Transformer model described in Dehghani et al. ( 2018 ) . We further improve this model by adding a latent variable that represents the persona and topics of interests of the writer for each training example .
In this study , we investigate the use of electromagnetic coupling for unifying the transformers used for the simultaneous transmission of energy and information . We studied the transmission of energy and information by wearing a unified transcutaneous transformer . In this system , it is important to minimize the electromagnetic coupling between the energy and information transmissions .
The invention discloses a method for compensating a dead area of a universal frequency transformer . The method comprises the following steps of : ( 1 ) acquiring three-phase current signals IU , IV and IW of a motor , and transmitting the signals to a digital signal processor to perform digital filtration by adopting a hardware filter ; and ( 2 ) performing linear processing on the compensation magnitude according to the magnitude and positive and negative of the detected current , namely when the current of one phase is positive , correspondingly increasing a dead area time to the switch-on time of an upper bridge arm Qx of an inverting bridge corresponding to the current , and correspondingly reducing a dead area time from the switch-on time of a lower bridge arm Qy of the inverting bridge corresponding to the current ; and when the current of the phase is negative , correspondingly reducing a dead area time from the switch-on time of the upper bridge arm Qx of the inverting bridge corresponding to the current , and correspondingly increasing a dead area time to the switch-on time of the lower bridge arm Qy of the inverting bridge corresponding to the current .
Some researchers try to enhance models ’ language comprehension ability by employing the pre-trained language models , but they neglect the importance of external knowledge in specific tasks . In this paper , we propose a novel universal transformer-based architecture for dialogue system , the Multiple Knowledge Syncretic Transformer ( MKST ) , which fuses multi-knowledge in open-domain conversation . Firstly , the model is pre-trained on a large-scale corpus to learn commonsense knowledge .
Through the experiment the proposed approach has been proved to be better in both semantic meaning and quality than the existing popular algorithms of keyword extraction . In addition , we propose to use contextual features from bidirectional transformers to automatically label short-sentence corpus with keywords and keyphrases to build the ground truth . This process avoids the human time to label the keywords and do not need any prior knowledge .
Both design methods are applied to two sample high voltage transformers . The measured performances of the as-built transformers highlight the limitations in using the conventional method , which are overcome using the reverse approach .
The auxiliary isolating switch comprises a supporting frame , a motor , a lead screw , a static contact , a movable contact component , an auxiliary switch , a cam , a first relay JD1 and a second relay JD2 , the motor is positioned on the supporting frame , the lead screw is supported on the supporting frame and driven by the motor , the static contact is positioned on the supporting frame , the movable contact component is composed of a movable contact support and a movable contact fixed on the movable contact support , the auxiliary switch is provided with normally-open and normally-closed contact switches K1 and K2 , the cam is supported on the supporting frame and used for driving the auxiliary switch to act , and the first relay JD1 and the second relay JD2 are arranged on the supporting frame . The auxiliary isolating switch is simple , reasonable and unique in structure , safe and reliable in running , low in cost and especially suitable for CRH5-type motor train unit traction auxiliary current transformers and is used for electric isolating of the auxiliary current transformers .
For grid optimization , DSOs have implemented dynamic voltage control in substations , adapted the grid structure for renewables , applied the “ N minus zero ” rule for planning grid operation and used the capability of renewable energy systems to provide reactive power . For grid expansion , DSOs have installed voltage regulators , voltage-regulated distribution transformers , substations used exclusively for renewables , and express feeders that connect substations with the generation centers . This practical experience should also prove relevant for other countries currently planning the expansion of distributed renewable energy systems .
This paper presents a control structure of Solid-State Transformers for three-phase AC/AC , to reduce the required size of the capacitors with load unbalanced compensation capability . The structure discussed in this paper consists of an AC/DC converter based on cascaded H-bridge ( CHB ) converters , isolated DC/DC converters , and a DC/AC inverter .
Smart-grid-enabled transformers have unique challenges in rural settings . One such challenge is that of effective and reliable communication of data .
However , in the case of a magnetic component , it is desirable to minimize the use because of the reduction of the traveling distance due to the increase in weight and volume and the insulation breakdown between the transformer windings due to vibration . Therefore , this paper proposes a dc-link voltage-variable integrated system that uses OBC and LDC transformers in common . The proposed circuit is divided into four operation modes .
The device has the size of the structure of a panel of the electrical accessory , and is mounted by using an electrical accessory mounting box . The circuit structure of the device comprises : a network processor , for controlling and processing the network application function of the network access device ; a wireless receiving and sending circuit , connected to the network processor and used for amplification , sending , and receiving of a wireless signal ; multiple network isolation transformers , provided between a wired network interface and the network processor of the network access device ; a power supply module ; and multiple toggle switches for setting the working mode . The network access device is encapsulated into a socket form of the electrical accessory , and is mounted at the mounting position of the electrical accessory of building facilities .
With the application of sensor networks in industrial , commercial , and medical areas , sensor operating system should be revisited to handle dependability problem . In this paper , we present a novel component-based sensor operating system , MTOS ( Micro-Transformers Operating System ) , which supports adaptive dynamic reconfiguration for resource-constrained sensor nodes to achieve dependability . Through error-driven reconfiguration and change-driven reconfiguration , MTOS can push a system into a stable state in which errors can be recovered with minimized side effects and changes can be safely made without interruption of normal services .
Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER Visualizing and Measuring the Geometry of BERT What Does BERT Learn about the Structure of Language? Are All Languages Created Equal in Multilingual BERT? Identifying Necessary Elements for BERT's Multilinguality Monitoring and early warning of new cyber-telecom crime platform based on BERT migration learning Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity. Classification of Fake News by Fine-tuning Deep Bidirectional Transformers based Language Model Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker L2RS: A Learning-to-Rescore Mechanism for Automatic Speech Recognition The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents Multi-Task Deep Neural Networks for Natural Language Understanding Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond Modeling Political Framing Across Policy Issues and Contexts Poor Man's BERT: Smaller and Faster Transformer Models Using Large Pretrained Language Models for Answering User Queries from Product Specifications Measuring and Reducing Non-Multifact Reasoning in Multi-hop Question Answering Clinical XLNet: Modeling Sequential Clinical Notes and Predicting Prolonged Mechanical Ventilation Sentiment Analysis Using Autoregressive Language Modeling and Broad Learning System Implementation of Simulink in Protection of Transmission Lines Attending to Mathematical Language with Transformers Electricity-stealing-preventing electric power system Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog Universal Transforming Geometric Network Pull distribution combined transformer that holds concurrently Latent Universal Task-Specific BERT Development of Unified Transcutaneous Transformer for Energy and Information Transmission for a Totally-Implantable Artificial Heart Method for compensating dead area of universal frequency transformer Multiple Knowledge Syncretic Transformer for Natural Dialogue Generation Self-Supervised Contextual Keyword and Keyphrase Retrieval with Self-Labelling Power Transformer Analytical Design Approaches Auxiliary isolating switch Innovative measures for integrating renewable energy in the German medium-voltage grids Unbalanced Load Compensation for Solid-State Transformer Using Smoothing Capacitors of Cascaded H-Bridges as Energy Buffer Communication with smart transformers in rural settings Integration of OBC and LDC Using Adaptive DC Link Voltage Socket-form network access device of electrical accessory Achieving Dependability through Dynamic Reconfiguration in Sensor Operating Systems
35480cc277b94dd653721ebc95d3b8d9cde6a46a afd110eace912c2b273e64851c6b4df2658622eb 335613303ebc5eac98de757ed02a56377d99e03a 3f791e19640bb79ff8727d0c8e204ea918172daa 0b9c41ec71ce9249e3e3ad718fff830b4129d71d e8613e20bd7c7b62035d507cbb1d4640a1fbc6a7 3af5a41e43158a75bf7a8bb3f9517edc4163b3ca a625de2fe477f169b49138c3307a14f8ef0e1093 73821c32df56bf077aeec940b189275ffe204599 7eaf9a7baa66ab6edf7d739d93e2e95f964c222f d3066ec95113636ca546cf4339772fbd495c27e4 b1dc77a526d6ad33de6e0bae148ab4875f5f6f0e 2b9955bc08fc5f4ddba73082ddabcfaabdbb4416 bd926efc4379488f0e3af047b8d533fd5059db24 dc76cd9a9075b8bbc5190ad8e4dc5a74a74d73ea 42f0bae2dacba44e9b5d8f050da3cbe41b9fc437 bf703a692a39687563c82bcc623eb55d3fbfccee 77d862c93a3e6f1d9663cdb7f3cae44412b84e2e 537a9ca1b98c58fc77cb175e049433b700c3404a f4af51de882598465bf2e9e508957c2136a53f43 f4e0723e048941ea73c77a7c69dbb731ef8de750 3cdaa387b08306012a28a1746d59f09941b7ed82 1e19301ec7e1a740375b663d85ed371078045c4e 8f8504aa422cada0f95a44255f804f1d79067b73 46239ef4b4d7a355c3158b4df058690c5eb10066 61596948c740ac61c225a8441e49e14b6c08e197 446e4ce32e36877e1670fed675d711a639e3d992 6f6d6ab4d9b7fbc311f192c12e17c0e4b775eecc ebc84a51f5c92dce64e77c9ac1b4a56ba6ada103 f913cd7a50ed63f20b13062280223c4e0d80081c da74dadf5ff7e62c10f2b01923a59a2051e396b3 12c7c4ce925a82b2146e497ad807302f8c2d58a8 0bfb1ea376db85e5715e0a302565d7824ca48287 06a80f9e24d8cee08b834c9db21b7d8ea9146746 f7216aa92af03dff18a7a1f452cea0f7da408e3d cdaa1c47e371bdee5e36e94233d649ffa3603eef
["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science","Engineering"] ["Computer Science"] ["Computer Science","Political Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Biology","Chemistry"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Engineering"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"]
2019 2019 2019 2020 2020 2020 2019 2018 2020 2019 2019 2019 2019 2019 2020 2020 2020 2019 2019 2018 2015 2020 2019 2015 2019 2009 2010 2020 2019 2007 2014 2020 2019 2014 2019 2013 2009
EMNLP/IJCNLP (1) ACL (1) ACL Transactions of the Association for Computational Linguistics ALTA BIBM WWW IECON EWDTS CSIE (1)
https://api.semanticscholar.org/35480cc277b94dd653721ebc95d3b8d9cde6a46a https://api.semanticscholar.org/afd110eace912c2b273e64851c6b4df2658622eb https://api.semanticscholar.org/335613303ebc5eac98de757ed02a56377d99e03a https://api.semanticscholar.org/3f791e19640bb79ff8727d0c8e204ea918172daa https://api.semanticscholar.org/0b9c41ec71ce9249e3e3ad718fff830b4129d71d https://api.semanticscholar.org/e8613e20bd7c7b62035d507cbb1d4640a1fbc6a7 https://api.semanticscholar.org/3af5a41e43158a75bf7a8bb3f9517edc4163b3ca https://api.semanticscholar.org/a625de2fe477f169b49138c3307a14f8ef0e1093 https://api.semanticscholar.org/73821c32df56bf077aeec940b189275ffe204599 https://api.semanticscholar.org/7eaf9a7baa66ab6edf7d739d93e2e95f964c222f https://api.semanticscholar.org/d3066ec95113636ca546cf4339772fbd495c27e4 https://www.semanticscholar.org/paper/658721bc13b0fa97366d38c05a96bf0a9f4bb0ac https://www.semanticscholar.org/paper/160563abbd75265b19afc8b4169bab9e1eb33d97 https://api.semanticscholar.org/b1dc77a526d6ad33de6e0bae148ab4875f5f6f0e https://api.semanticscholar.org/2b9955bc08fc5f4ddba73082ddabcfaabdbb4416 https://api.semanticscholar.org/bd926efc4379488f0e3af047b8d533fd5059db24 https://api.semanticscholar.org/dc76cd9a9075b8bbc5190ad8e4dc5a74a74d73ea https://api.semanticscholar.org/42f0bae2dacba44e9b5d8f050da3cbe41b9fc437 https://api.semanticscholar.org/bf703a692a39687563c82bcc623eb55d3fbfccee https://api.semanticscholar.org/77d862c93a3e6f1d9663cdb7f3cae44412b84e2e https://api.semanticscholar.org/537a9ca1b98c58fc77cb175e049433b700c3404a https://api.semanticscholar.org/f4af51de882598465bf2e9e508957c2136a53f43 https://api.semanticscholar.org/f4e0723e048941ea73c77a7c69dbb731ef8de750 https://api.semanticscholar.org/3cdaa387b08306012a28a1746d59f09941b7ed82 https://api.semanticscholar.org/1e19301ec7e1a740375b663d85ed371078045c4e https://api.semanticscholar.org/8f8504aa422cada0f95a44255f804f1d79067b73 https://api.semanticscholar.org/46239ef4b4d7a355c3158b4df058690c5eb10066 https://api.semanticscholar.org/61596948c740ac61c225a8441e49e14b6c08e197 https://api.semanticscholar.org/446e4ce32e36877e1670fed675d711a639e3d992 https://api.semanticscholar.org/6f6d6ab4d9b7fbc311f192c12e17c0e4b775eecc https://api.semanticscholar.org/ebc84a51f5c92dce64e77c9ac1b4a56ba6ada103 https://api.semanticscholar.org/f913cd7a50ed63f20b13062280223c4e0d80081c https://api.semanticscholar.org/da74dadf5ff7e62c10f2b01923a59a2051e396b3 https://api.semanticscholar.org/12c7c4ce925a82b2146e497ad807302f8c2d58a8 https://api.semanticscholar.org/0bfb1ea376db85e5715e0a302565d7824ca48287 https://api.semanticscholar.org/06a80f9e24d8cee08b834c9db21b7d8ea9146746 https://api.semanticscholar.org/f7216aa92af03dff18a7a1f452cea0f7da408e3d https://api.semanticscholar.org/cdaa1c47e371bdee5e36e94233d649ffa3603eef