The Generation of Textual Entailment with NLML in an Intelligent Dialogue system for Language Learning CSIEC
This research report introduces the generation of textual entailment within the project CSIEC (Computer Simulation in Educational Communication), an interactive web-based human-computer dialogue system with natural language for English instruction. T…
Authors: Jiyou Jia
The Generation of Textua l Entailment with NLML in an Intelligent Dialogue system for La nguage Learning CSIEC Jiyou Jia Departm ent of Educati onal Tech nology Peking University Beijing, 100871, China jjy@pku.edu.cn Abstract This research report introduces the generation of textual entailment within the project CSIEC (Computer Simulation in Educational Com - munication), an interactive web-based hum an- computer dialogue system with natural lan- guage for English instruction. The generatio n of textual entailment (GTE) is critical to the further improvement of CSIEC project. Up to now we have found few literatures related with GTE. Simulating the process that a hu- man being learns English as a foreign lan- guage we explore our naïve ap proach to tackle the GTE problem and its algorithm within the framework of CSIEC, i.e. rule annotation in NLML, pattern recognition (matching), and entailment transformation. The time and space complexity of our algorithm is tested with some entailment examples. Further works in- clude the rules annotation based on the Eng- lish textbooks and a GU I interface for normal users to edit the entailment rules. 1 Problem Description: Generation of Textual Entailment (GTE) and Recog- nition of Textual Entailment (RTE) In daily human language we can often infer (entail) one t ext f ragment from anothe r one, for ex ample “ what is the boo k’s price? ” and “ how much is the book? ”. We adopt a definition of textual entailment as a d irecti onal relat ionshi p bet ween pa irs o f te xt expressions, denoted by T (the entailing text), and H (the entailed hypothesis). It i s considered that T entails H if th e meaning o f H can be inferred from the meaning of T, as would typ ically be inferred by people (Dagan et al., 2006 ). This somewhat i nfor- mal definition is b ased on (and assumes) co mmon human understanding of langu age as well as com- mon background know ledge. In some works (as Lin, et al. 2001) the entailment is called inference. That T entail s H means al so T infers H . So in this paper we treat these two terms as the same. This definit ion ca n be des cribed wi th a formula: T H, where T is the entailing text and H is t he entailed hypothesis. T H is true i f T entails H, and this relationshi p is called an entailment or inference rule. If both T H and H T are true, w e call T is equivalent to H, or T H. The following are simple examples of text en- tailment and equivalence frequ ently used in daily life and in the system CSIEC: (E1) What is the price of the b ook? How much is the b ook? (E2) The student’s name is Zhang. The student is Zhang. (E3) I study in Beijing Univ ersity. I attend Beijing Univers ity. (E4) English is his mother la nguage. He can speak English. Two questions emerge with the tex t entailments: The fi rst is: given a text T, to cal culat e what c an be inf erred f rom T. The second is: given two texts T and H, to de- termine whether T H is true or false. We call the first questi on Generation of Textual Entailment (abbreviation: GTE ), the second one Recognition of Textual E ntailment (abbreviation : RTE ). This paper attempts to solve GTE problem within the framework of CSIEC pro ject. 2 The Significa nce of GTE to CSIEC CSIEC (Computer Simulation in Educational Communi cation) i s an intera ctive we b-base d hu- man-computer dialo gue system with natural lan- guage for English instruc tion (Jia, 2004, a). It has been put into free usage in Internet an d has also been applied in English class. Despite of its achievements currently there are still some user requirements w hich haven’t been fulfilled well, for example, the system’s stro ng ability of natural lan- guage understating and generation , which is the fatal factor influencing the human-computer com- munication. The generation of textual entailment plays an importan t role to the m. At first the GTE is related to the redundancy of the facts database. The CSIEC can collect the user facts from the user exp ressions into th e form of NLML, and save these facts into the table “user fact s” in NLDB. Th e facts about t he chatti ng robo t are also treated by this way, i.e. the n arrative dec- larative sentences about the robot are stored with the form NLML into one separate table in NLDB. But how to deal with the facts which can be in- ferred from other facts? If all they are stored in the database, the redundancy of the knowledge data- base will be greatly increased. For example: if the user inputs “ I am an English teacher in Beij ing Univer sity. ” at first interaction with the robot, this fact will be recorded in the NLD B. Late r i f he /sh e in pu ts “ I teach English in Beijing Univers ity. ” again, the robot should be able to recognize th at the user is repeating or e mphasiz- ing a fact abou t him(her)self which can b e inferred from the previous fact, and need not to add a n ew record “ I teach English in Beijing University. ” into the NLDB. Moreover, the robot should be able to generate the new sentence “ You teach English in Beijing University. ” as a response to the user’s in- put. Altho ugh it is jus t a repeti tion of the u ser’s input, it shows the ro bot’s ability of logical infe- rence, and can be regarded as a m eaningfu l re- sponse. Secondly the GTE is also related to the i mple- mentat ion of guided ch atting on a given to pic. F or example in the guided discourse “ Sa lesman and customer ” if the computer knows “ what is its price ?” equals “ how mu ch is it ?”, and “ may I help you? ” equals “ can I help you? ”, we need no t to write all the equiv alent expression s into the scena- rio script. Thirdly the entail ment calculation can also con- tribute to the question answ ering. In CSIEC the robot gets answers to the users’ questions through scrutinizing the user facts database and the com- mon sense knowledge database. The user facts da- tabas e are enric hed thr ough t he i nteracti on bet ween the user and the robot. If the user wants to test the intelligence of the robot, he/sh e may ask a question such as “ Who am I ? ”. Ba sed on the us er fact “ I am an English teacher in Beijing University ” the robot can answer with: “ Y ou are an English teacher in Beijing University. ” If the user asks “W hat do I teach? ”, the answer could be readily obtained from the enta ilment: “ I tea ch English ”. At last the entailment generation can help the system’s adaptation to the user language level. Th e vocabulary and grammar skill of a language learn- er varies in every learning stage. Th us the response with different levels of vo cabulary and grammar skills should be generated corresponding to the learner’s linguistic leve l. For the beginners the sys- tem should respond with a simple sentence, whe- reas for the advanced users the robot can speak more complicated sentence with unfa miliar words. One example is that the middle school students can unders tand “ This problem is difficul t. ”, but ma y not unde rstand “ This problem is knotty. ”, bec ause they haven’t learned the word “ knotty ”. So the text entailment generation is significant and critical to the evolv ement of the CSIEC sy s- tem. Moreover, as the RTE is associa ted with the GTE, we believe the solution of GTE will help the solution of RTE a nd othe r relat ed prob lems such as question answering, information retrieval, etc. 3 Related Works to Solve GTE Through the literatu re survey we can’t find rela ted works specialized on the textual enta ilment genera- tion, although amounts of papers have presented the pioneer researches o f discovering the in fe- rences or checking the entailment relationship be- tween two texts. PASCAL recognizing textual entailment (RTE) challenge (http://www.pascal- network.org/Ch allenges/RTE) is organized to ex- plore what can be achieved in the area of RTE with current state-of-the-art tools. From its two past procee dings (Dagan e t al. 200 5; Ba r-Haim et al, 2006) we can hardly find the work on GTE. Lin and Pantel (2001) could be the first attempt to discover inference knowledge from a large cor- pus of text. They intro duced the Extended Distri- butional Hy pothesis, w hich sta tes that p aths in dependency trees have similar meanings if they tend to connect si milar sets of words. Treating paths as binar y relat ions, their unsuper vised al go- rithm, DIRT (D iscovering Inference from Text), can generate inference rules by searching for simi- lar paths. A chart parser Minipar is used in DIRT to get the dependency tree. They extracted 7 mil- lion inference rules from the parse trees, among them 231,000 are unique . Howeve r, the hum an linguistic experts should still work to check which of the inference rules are correct one by one. The accuracy rate ranges fro m 0% to 92.5%. The famous logical programming sy stems such as Prolog and LISP can make inferences according to the logical rule s. However, the laborious trans- formation from natural language into exact logical language and v ice versa see ms to be only done by the logician experts. So in our project we attempt to directly teach the co mputer how to und erstand the textual entail ment rules and how to make infe- rence according to the entailment rules with the notation of NLML and NLOMJ, just as the English teacher teaches the students how to learn the sen- tence patterns and transformation rules, and how to apply them in actual langu age expression. 4 Our Naïve Approach to S olve GTE with NLML 4.1 Principle: rule annotation, pattern matching and entailment transformation The entailment problem emerges with the n atural language acquisition. Recall how the teachers taught us English and how we learned English as a second language. To get the entailment of example E1 in Section 1 , we need just to chang e some words, and the other phrases remain unchanged. Apparently only remembering this special ru le is not too useful. We can describe the equivalence more generally, as the teacher taught us: (R1) What is the price of ? How much is ? We call the < someth in g > a pseudo variable in the rule, and “ a book ”, “ those p ens ”, and so on, the variable’s value. The entailment generation is ac- tually the replacement o f the pseudo v ariables in the rule right with their concrete values obtained by matching the given text with the rule left. From the example E2 we shou ld obtain a more gen era liz ed e nta il men t ru le: (R2) < person> ’s name be < person > be . In this rule there are two pseudo variables. “ ” represents a person, such as a student. “ ” represents the concrete name, such as “ John ”, “ Bill Clinton ”. “ be ” represents the con- crete copula fo rm, such as “ was ”, “ is ”, “ has been ”. This rule doesn’t co ntain any circu mstance limi- tation, like time and place circu mstances. This means it can be applied in any circumstance. With this generalized rule we can get the following text entailment pair: The student’s name was John five years ago. The student was John five years ag o. We have learned these entailment rules by heart in the language course and can apply the m uncon- sciously, though up to now we haven’t discovered how our brain cop es with such replacement. The rules a re act ually the gramm atical and logi cal rules we learned from English textbooks. We sh ould rewrite them into a form which the computer can understand and use. This is the first step by letting the comput er learn the infe rence rules. After the storage of the entailment rules the next step is generating th e entailment for a given text. We can match the text with the rules in the data- base one by one, and find which one has the sa me structure as the given tex t. A question occurs: what does the same structure mean between two expres- sions? We remind again our grammatical know- ledge about English. The same structure means: they both have t he sam e mood (d eclara tive, inter- rogative, imperative or exclamative), and the same sentence structure. For example the left side of the rule (R1) is a question, and its sente nce structure is “subject + be + nou n phrase predicate”. After finding the appro priate rule for the given text, the third step is replac ing the corresponding pseudo variables of the rule right (enta ilment) with their concrete values, and setting the verb in the appropriate form based on the given text, main ly on the tense of the given te xt and the actual subject. The values can be retrieved during the matching proce ss. So our principle to generate textu al entailment is to describe the en tailment rules with the ap propri- ate form, to search the matched rule for a given text, and then to replace the pseudo variables in the rule ri ght with the act ual valu es and to se t the verb with the suitable form. We call t he three steps rule annotation, pattern matching and entailment trans- form ation, respe ctivel y. 4.2 Rule annotation: NLML with pseudo va - riables NLML, as defined in (Jia, 2004, b) is a mixture of phrase tree structure and dependency tree structu re, and a detailed syntactic and semantic analy sis of natura l la nguage text. Almost all lingui stic f eature s (words, part-of-speech, entity typ e, chunk tag, grammatical function tag, head word path) are in- cluded in NLML. All kinds of grammatically right expressions, e.g. phrases and sentences, with dif- ferent complexity, voice and moods, can be clearly described by NLML. Thus it c an construct the ba- sis for further syntactical and semantic analysis. For exam ple it can be parsed i nto the object model of natural language expressions, NLOMJ (Jia et al., 2004), which is suitable for rule matching and ru le repl aceme nt. Limited by the pa per l ength we just gi ve the NLML of the example E1 left (“ what is the book ’s price ? ”): question simpl e relpron what be sing th ird pres ent is np art the < /prem > third sing price noun of art the third book sing noun We now introduce the entailment rule form and annotation with some examples. For the entailment rule (R1) left, we keep the subject and verb phrase “ what be the price ” as required pa ttern, change the object “ the b oo k ” of the prepo sitional phrase to a pseudo variable indexed as 1, so its NLML is: qu esti on si mple relpr on what be present s ing thir d< /per s> is np third price< /word> s ing noun of art the third pseudo variable 1 noun qu esti on si mple pseud o vari able 1 be query_adj < adv> ex tent how much abso< /grad > Starting from this rule we can generate the follow- ing text entai lment pairs: What was the pen’ price two years ago? How much was the pen two years ago? What has the price of the bus tickets in the capital Beijing been since 2007? How mu ch have the bus tickets in the capital Beijing been since 2007? For the rule R2: ’s name be < perso n > be , the ru le le ft ( tex t) NLML is: statement simpl e third name sing noun of art the < /prem > third pseudo variable 1 person sing noun be sing th ird pres ent is person” indicates the noun in the preposition al phrase modif ying the noun “ name ” must be an instance of the class “ person ”. What is a “ perso n ” then? “ A student ”, “ a teacher ”, etc. are persons. Th is rela- tionship between an occupation an d a person can be retrieved from WordNet. “ Your sister” , “ his father ”, etc. are persons. This family relationship can also be retrieved from the WordNet. Thu s the classification of a noun phrase in to the category “ person ” can be realized with WordNe t. The tag “ name” in the NLML has defined the predicate must be a person name, thus the new tag “ category ” is not nec essary. The entailm ent NLML is: statement simple pseud o vari able 1 < verb_chan ge/ > be np pseudo variable 2 Here th e tag “ statement si mple pseudo variable 1 active verb present sing first study prep_phrase in third< numb>si ng address Beijing group third sing pseudo variable 2 noun statement simpl e pseudo var iable 1 < verb_change/ > active verb_object attend p seudo variable 2 The entailment requires the kernel verb mu st be “ attend ” whose form depends on the given text, so the tag “ is somebody’s mother language somebody can speak . Its text NLML is: statement simpl e third sing pseudo variable 1 language noun be sing third present is np possessive pseudo variable 2 noun mother language thir d< /per s> s ingnoun statement si mple pseud o vari able 2 verb_object modal< /tense> s ing first can speak infi pseudo variable 1 This entailment states the verb part must be “ can speak ”, so needs not the tag “< verb_change/> ”. By replacement th e pronoun shou ld be changed fro m its genitive (possessive) case to no minative case. 4.3 Pattern matching How to check if a given text is a concrete instance of the generalized model of the rule’s left? Sup- pose the given text is T, and the given text pattern (rule left) is P. As to all the examp les mentioned above wi th thi s “simpl e” compl exity (se ntenc es without conjunctio n like “ before ”, “ if ”), the algo- rithm o f the pattern matching is: Compare the mood of the T with the mood of P If they aren’t equal, T doesn’t match P Else Compare the subject of T with the subject o f P If the subject of T does not match the subject of P, T does not match P Else Compare the verb phrase of T with that of P If the former does not mat ch the latter, T does not match P Else Compare the circumstances of T with that of P If an y circ umstanc e in P can’ t fin d a matched circumstance in T, T does not match P Otherwise T does match P End Then more concretely, what is t he matching of the given subject to the given pattern subject? As a subject is actually a noun phrase used at the sen- tence beginnin g, the c ompariso n of t wo subje cts is in fact the comparison of two noun phrases. By the comparison of a concrete noun phrase with the noun phrase p attern, the content o f the attribute “ pseudo ” in the pattern is firstly checked. A not empt y cont ent m eans this patt ern no un p hras e doesn’t have any specification; therefore the checked noun phrase matches this pattern. After- wards the pse udo variable rep resented by th is pat- tern content will be s et the value of the N LML of the checked noun phrase. An empty content of the attribute “ pseudo ” in the pattern phrase means the re are specific re- quirements in the no un phrase. The given text should be checked more in detail with the pattern phrase . The matching of a given verb p hrase to the g iv- en pattern verb phrase requires both th e verb part matching and the matchi ng of other components. By the verb part matching at first the verb type and voice (active or passive) of the checked verb phrase will be com pared wit h that of t he patte rn phrase . If ei ther t he type or voic e isn’t equal to t hat of the pattern p hrase, this given verb p hrase doesn’t match t he pat tern verb ph rase. Ot herwi se the other parts in the verb ph rase will be further checked. If the verb parts of the verb ph rase match that of t he patte rn, t he othe r parts , e.g. predi cate, or objects, should be checked further. 4.4 Entai lment tr ansform ation If one given text matches the left pattern of the ru le, the pseudo variables in the pattern will be set the corresponding values, i.e. the NLML of the matched phrase. With them we can obtain the en- tailment of the given text according to the rule. The algorithm is: Replace the “ pseudo variable X ” (X is t he se- quence number of the pseudo variable, such as 1 and 2) in the NLML of the rule right with the c or- responding actual NL ML which have been ob- tained during the pattern matching process. If there is tag “ verb_change ” in the entailment NLML, transfer the attributes of the verb phrase according to subject phrase and the tense of the given text. Afte r the ve rb phras e tra nsform atio n we get the ultimate NLML of the entailment and then calcu- late the enta iled text after par sing it into NLOMJ. 4.5 GTE Algorithm and logical entailment Through the examples above we have show ed the procedu re of t extual entail ment ge neration. The algorithm for GTE can be su mmarized as: Pre process the text Parse the text into NL ML Parse the NLML in to NLOMJ Compare: compare the structure of this NLOMJ with th e rul es in ru le d atab ase If there is such a rule or rules match ing this one as text transform the entailment NLML and cal- culate then the entailment tex ts repeat Compare with the obtained en tail- ments to get deeper entailments Otherwise, employ the logical entailment algo- rithm to get all of i ts entailments. The logical e ntailmen t of a text is the en tailmen t which needs not any inference rule, but can be in- ferred according to the log ical reasoning of the common sense k nowledge. For example: hy - pernym entailment: Zhang is a student. Zhang is the , e.g. Zhang i s a per son . I have a dog I have a , e.g. I have an animal. The deeper entailment means the entailment of the entailment. To avoid repeating reversed entailment, we label each rule with an identification number, and specify the number of its reversed rule. For example if we label the rule R3 with the ID=3, and label its reversed rule R5 with ID=5, the R3’s re- versed rule ID is 5, and vice versa. (R5) somebody attends this university/institu te /college. Som ebody studies in a given university /institute/co llege. 4.6 Assistant authoring tool to edit the entail- ment rules: TEE Apparently it is too difficult for a normal English teacher to edit the entailment rules. We have de- signed a Java GUI, a so called TEE (Textual En- tailment Editor), to assist the no rmal user to edit the rule easily. The rule annotator needs not to re- member the NLML in details, but only input one example pair of text an d entailment. The TEE then will interactively guide the annotator to m ake some choices by just clicking on buttons, and finally get the entailment rule. At the end the annotator can check the rule with new texts. Of course the anno- tator should be good at English grammar. 5 Implementa tion and Co mplexity Pre- diction We are cooperating w ith English te achers to ma- nually build the entail ment rule database for the textual entailment occurr ing in the textbooks of schools and universities with TEE. An annotat or can write 10 rules in one hour. This method is la- borious; however, it is a reliable one. If it is impos- sible for the human being to write all of the rules implicit hidden in the giant moun ts of corpus, it is still plausible to write the rules taught in the Eng- lish courses from ele mentary school to univers ity, which can contribute much to improve the intelli- gence of CSIEC system. The entailments ru les and their test can be accessed in the CSIEC website. Besides the rules which can be easily retrieved from the Englis h textbo oks, we ar e also pl anning to use the resources available from Internet, for example, Sekine's Paraphrase Database ( http://nlp.cs.nyu .edu/paraphrase/ ), and TEASE ( http://aclweb.org/aclwiki/ind ex.php?title=TEASE ). The former includes a paraphrase database by Ha- segawa's method (Hasegawa et al. 2004) with 755 sets of paraph rases, and 3,865 phrases in to tal, which have been cleaned up by human annotator, as well as the parap hrase database by Sekin e's me- thod (Sekine 2005) with 19,9 75 sets of parap hrases, and 191,572 phrases in total, which have not been cleaned up by human. The latter (Szpektor et al. 2004) consists of 136 different templa tes, every of which is a set of entailmen t relations. To predict the co mplexity of the G TE algorithm dealing with more and more entailment rules, we make a test wi th 10, 000 rule s, am ong them 20 a re unique and the others are the same, what will no t reduce the comparison time. The generation of tex- tual entail ment for a given text costs 1 00ms or so time, almost the same as the testing with just 20 unique rules. But the memory cost is linearly pro- porti onal to th e rules num ber. The 10,000 rule s occupy about 300 Megabytes physical memory. 6 Discussion The underlying idea of our GTE algorithm is very naïve: the language teachers to ld us the sentence pattern and inference rules, w e learn by heart these rules one by one and apply the m in thinking and speaking. No almighty method can be given by the teacher and learned by us so that we can use it to all entailment generation. In language education, this is an o ld, trad itional and plausib le way . But to the best of our knowledge, no researcher in com- puter science and NLP has used it to m odel the GTE in computer. Maybe the algorithm complex- ity by dealing with the se emingly very large amount of rules is the main obs tacle (Stefik, 1995), .e.g. the 231,000 unique inference rules found in (Lin and Pantel, 2001). We will also face this problem, as the rule database grows. Additionally, the rule annotation with NLML and its machine interpretation will become more complicated. In this paper we just illustrated the GTE algorithm with the example of simple sen- tences with only one subject plus one verb phrase structure. More work should be done to solve the GTE of complex sentence s with subordinate sen- tences. We begin the research of GTE from our in terac- tive language le arning project CSIEC with the ob- jectives to reduce the facts redundancy, to generate reasonable and diverse responses, and so on . So the evaluation of the application results should be im- plemented in the futur e. Moreover , we will attempt to use the G TE approach to tack le other hard prob- lems in NLP, such as RTE, question answering and infor mation retri eval, etc. Acknowledgments We thank the support to our projects from Ministry of Education China, Beijing University, and Edu- cation Co mmittee of Capital Beijing. References R. Bar-Haim, I. Szpecktor, an d O. Glickman. 20 05. De- finition and analysis of inte rmediate entailment le- vels. Proceedings of the ACL Workshop on Empirical Modelling of Semantic Equivalence and Entailment , pp. 55–60. Association for Computational Linguis- tics. R. Bar-Haim, I. Dagan, B. Dolan, L. Ferro, D. Giampic- colo, B. Magnini and I. Szpektor. 2006. The Second PASCAL Recognising Textual Ent ailment Chal- lenge. Proceedings of the Second PASCAL Chal- lenges Workshop on Recognising Textual Entailment. J. Bos & K. Markert. 20 05. Recognising Textual En- tailment with Logical Inference. Proceedings of EMNLP 2005. R. Braz, R. Girju, V. Pu nyakanok, D. Roth, and M. Sammons. 2005. Knowledge Representation for Se- mantic Entailment and Question-Answering. IJCAI- 05 Workshop on Knowledge and Reasoning for Ans- wering Q uesti ons. I. Dagan, O. Glickman and B. Magnini. 2005. P roceed- ings of the PASCAL Challenges Workshop on Recog- nising Textual Entailment. LNCS, Vol. 3944. C. Fellbaum. 1998. Ed.: WordNet: An Electronic Lexi- cal database. MIT Press. M. Geffet and I. Dagan. 2004. Feature Vector Quality and Distributional Similarity. Proceedings of the 20th International Conference on Computational Linguis- tics (COLING) . M. Geffet and I. Dagan. 200 5. The Distributional Inclu- sion Hypotheses and Lexical Ent ailment. ACL 2005, Michigan, USA. U. Hahn and K. Schnattinger. 1998. Towards text know- ledge engineering. Proceedings of AAAI-98. pp. 524- 531. Menlo Park, Californi a. A . H a g h i g h i , A . Y . N g , C . D . M a n n i n g . 2 0 0 5 . Robust Textual Inference via Graph Matching. HLT- EMNLP 2005. T Hasegawa, S. Sekine, R. Grishman. 2004. Discover- ing Relations among Named Entities from Large Corpora , ACL-04 . J. Jia. 2004, a. CSIEC (Computer Simulator in Educa- tional Communication): A Virtual Context- Adaptive Chatting Partner for Foreign Language Learners (in English). In Kinshuk, etc. (Eds.), Proceedings o f ICALT 04 , P.690-692. New York: IEEE Press. J. Jia. 2004, b. NLML - a Markup Language to Describe the Unlimited English Gramm ar. CoRR cs.CL/0404018: (2004). J. Jia, Y. Ye, K. Mainzer. 2004. NLOMJ--Natural Lan- guage Object Model in Jav a, in Z. Shi, Q. He (Eds.), Intelligent information processing II. Proceedings of ICICP 04 , P.201-209. New York: Springer. C.H.A. Koster. 1991. Affix Gramm ars for natural lan- guages. Attribute Grammars, Applications and Sys- tems, International Summer School . Lecture Notes in Computer Science, volume 545. Springer. D. Lin. 1998. Dependency-based evaluation of MINIPAR. Proceedings of the Workshop on Evalua- tion of Parsing Systems at LREC. D. Lin, P. Pantel. 2001. Discovery of inference rules for question-answering. Natural Language Engineering , 7: 343- 360 Ca mbr idge Un iver sity Press. B. MacCartney, T. Grenager, M. de Marneffe, D. Cer and C. D. Manning. 2006. Learning to Recognize Features of Valid Textual Entailments . HLT-NAACL 2006. V. Pekar. 2006. Acquisition of Verb Entailment from Text . HLT-NAACL 2006. R. Rai na, A. Y. Ng, a nd C. Manning. 2005 . Robu st textual inference via learning and abductive reason- ing. Twentieth National Conference on Artificial In- telligence (AAAI-05). L. Romano, M. Kouylekov, I. Szpektor, I. Dagan an d A. Lavelli. 2006. Investigatin g a Generic Paraphrase- based Approach for Relation Extraction. EACL 2006. V. Rus, A. Graesser and K. Desai. 2005. Lexico- Syntactic Subsumption for Textual Entailment. RANLP 2005. S. Sekine. 2005. Au tomatic Paraphrase Discovery based on Context and Keywords between NE Pairs. IWP-05. Y. Shinyama, S. Sekine, K. Sudo and R. Grishman, A u- tomatic Paraphrase Acquisition from News Articles, HLT-02. M. Stefik. 1995. Introduction to Knowledge System s. San Francisco, CA: Morgan Kaufm ann.
np pseudo variable 2 name third < numb>sing
Original Paper
Loading high-quality paper...
Comments & Academic Discussion
Loading comments...
Leave a Comment