حسن شامحمدی - دانشکده فنی و مهندسی
دانشکده فنی و مهندسی
گروه آموزشی مهندسی کامپیوتر
اطلاعیه دفاع از پایان نامه کارشناسی ارشد در رشته مهندسی کامپیوتر گرایش هوش مصنوعی
عنوان:
تشخیص نقلبهمضمون با استفاده از تکنیکهای یادگیری عمیق
استاد راهنما:
دکتر میرحسین دزفولیان
استاد مشاور:
دکتر محرم منصوری زاده
اساتید ممتحن:
دکتر حسن ختنلو
دکتر مهدی عباسی
پژوهشگر:
حسن شاهمحمدی
زمان:
یکشنبه 30/10/1397 ساعت 14:00
مکان:
سمینار 2 دپارتمان برق (سالن مهندس مرحوم خانمحمدی)
Bu-Ali Sina University
Faculty of Engineering
Department of Computer Engineering
Thesis submitted for Master of Science in computer Engineering-Artificial Intelligence
Title:
A novel method for point cloud analysis based on deep learning techniques
Supervisor:
Dr. Mir Hossein Dezfoulian
Advisor:
Dr. Muharram Mansoorizadeh
Judges:
Dr. Hassan Khotanlou
Dr. Mahdi Abbasi
Author:
Hassan Shahmohammadi
January 20, 2019
تشخیص نقلبهمضمون با استفاده از تکنیکهای یادگیری عمیق
تشخیص نقلبهمضمون یکی از مسائل مهم در حوزه پردازش زبانهای طبیعی است. نقلبهمضمون به جملات یا عباراتی اشاره میکند که معنی و مفهوم یکسانی را به خواننده منتقل میکنند اما ساختار و کلمات آنها با هم متفاوت است. این مسئله کاربردهای فراوانی در حوزه پردازش زبانهای طبیعی دارد. ازجمله این کاربردها میتوان به استفاده آن در خلاصهسازی متن، ترجمه ماشینی، سیستمهای پرسشوپاسخ، تشخیص سرقت ادبی و موتورهای جستجو اشاره کرد. در این پژوهش، ابتدا مسئله با چندین روش مرسوم مانند وزندهی TF-IDF و استفاده از طبقهبندهایی همچون ماشین بردار پشتیبان، حل و ارزیابی شده است. سپس با استفاده از نتایج بدست آمده از این روشها، یک مدل جدید برای تشخیص نقلبهمضمون ارائه شده است. مدل پیشنهادی را میتوان به دو بخش تقسیم نمود. در بخش اول که مسئله با تکنیکهای یادگیری عمیق حل میشود، جملات پس از عبور از مرحله پیشپردازش، با استفاده از تکنیک پنهانسازی GloVe به بردارهایی عددی تبدیل میشوند. خروجی این لایه پنهانسازی سپس به یک شبکه Bi-LSTM برای بیان کردن کل جمله داده میشود. پس از اتمام آموزش مدل، خروجی این شبکه بهعنوان ویژگیهای استخراج شده برای هر جمله در نظر گرفته میشوند. در بخش دوم، یک سری ویژگی دستی برای بیان کردن میزان شباهت معنایی بین دو جمله معرفی میشوند. از میان این ویژگیها، تعدادی از آنها جدید بوده و برای اولین بار در این پژوهش معرفی شدهاند. مدل پیشنهادی از ترکیب ویژگیهای بدست آمده در این دو بخش حاصل میشود. دو مجموعه داده با نامهای MSRP و Quora برای ارزیابی مدل پیشنهادی در نظر گرفته شدهاند. نتایج مدل برای مجموعه داده MSRP نشان میدهد که این مدل تقریبا از تمام پژوهشهای انجام شده، کارایی بهتری از نظر صحت و f-measure را کسب میکند. نتایج ارزیابی مدل برای مجموعه داده Quora نیز کارایی قابل قبول و قابل مقایسهای با سایر پژوهشهای انجام شده روی این مجموعه داده را نشان میدهد. به طوری که مدل پیشنهادی جزء 24 درصد برتر روشها از میان بیش از 3000 تیم در سایت Kaggle است. نتایج ارزیابی همچنین نشان میدهد که مدل پیشنهادی برای مجموعه دادههایی با تعداد نمونههای کم، کارایی بهتری در مقایسه با سایر مدلهای جدید دارد.
Paraphrase detection using deep learning techniques
Paraphrase detection is one of the fundamental tasks in the area of natural language processing. Paraphrase refers to those sentences or phrases that convey the same meaning but use different wording. It has a lot of applications. For instance, it could be used in machine translation, text summarization, QA systems, plagiarism detection, and search engines. In this research, we fist employed several common classification and feature extraction methods to deal with the problem. Models used in this step are simple, for example TF-IDF term weighting and SVM classifier are employed. In the second step, advantages and disadvantages of each simple model is scrutinized. Then, a new model is introduced based on the conclusion of the results of the simple models. The new model can be divided into two parts. In the first one, each sentence pass through a preprocessing step and then its constitutive words convert to their numerical representation using GloVe word embedding. The output of the GloVe word embedding is then fed into a Bi-LSTM neural network to encode the whole sentence by leveraging its word vectors. In the second part, three sets of handcrafted features are used to measure the similarity between each pair of sentences. Some of these features are new and introduced in this research for the first time. The new model is formed by combining the handcrafted features and the output of Bi-LSTM network. Two datasets are used to evaluate the model namely, MSRP and Quora. The result of evaluating the model on MSRP shows that it outperforms almost all the previous works in terms of f-measure and accuracy. The evaluation results on Quora also shows comparable results among other works. On Quora-question pair competition launched by Kaggle, our model ranked 24% top solutions among more than 3000 teams. The results of evaluation for both datasets shows that our model achieves the best relative results when the training data is not huge. In other words, it can generalize well despite the lack of adequate training data for deep models.
Paraphrase detection using deep learning techniques
Paraphrase detection is one of the fundamental tasks in the area of natural language processing. Paraphrase refers to those sentences or phrases that convey the same meaning but use different wording. It has a lot of applications. For instance, it could be used in machine translation, text summarization, QA systems, plagiarism detection, and search engines. In this research, we fist employed several common classification and feature extraction methods to deal with the problem. Models used in this step are simple, for example TF-IDF term weighting and SVM classifier are employed. In the second step, advantages and disadvantages of each simple model is scrutinized. Then, a new model is introduced based on the conclusion of the results of the simple models. The new model can be divided into two parts. In the first one, each sentence pass through a preprocessing step and then its constitutive words convert to their numerical representation using GloVe word embedding. The output of the GloVe word embedding is then fed into a Bi-LSTM neural network to encode the whole sentence by leveraging its word vectors. In the second part, three sets of handcrafted features are used to measure the similarity between each pair of sentences. Some of these features are new and introduced in this research for the first time. The new model is formed by combining the handcrafted features and the output of Bi-LSTM network. Two datasets are used to evaluate the model namely, MSRP and Quora. The result of evaluating the model on MSRP shows that it outperforms almost all the previous works in terms of f-measure and accuracy. The evaluation results on Quora also shows comparable results among other works. On Quora-question pair competition launched by Kaggle, our model ranked 24% top solutions among more than 3000 teams. The results of evaluation for both datasets shows that our model achieves the best relative results when the training data is not huge. In other words, it can generalize well despite the lack of adequate training data for deep models.
Hassan Shahmohammadi
| POB: 6931618352, Unit8, Floor 5, Shohada Street (between Shohada and Khayyam square), Ilam city, Ilam, Iran.
|
Education and Qualifications
· BSc | Software engineering , University of Kurdistan, September 2012 – August 2016 Thesis title: An android application to full control personal computers via Wi-Fi connection.
|
· MSc | Artificial intelligence, University of Bu-Ali Sina, September 2016 – December 2018 (expected) Thesis title: Paraphrase detection using deep learning techniques. |
___________________________________________________________________________________
Research Interests
· NLP: QA and recommender systems, short text analysis, semantic text representations, word embedding, sentiment analysis, topic modeling, recognizing textual entailment, paraphrase identification.
· ML: Deep Learning models, ensemble techniques.
· DIP: Image tagging, image captioning, image classification.
Awards and Honors
· 2008, Attending in Final Level of 4th Iran National Astronomy and Astrophysics Olympiad
Publication
· "An Extensive Comparison of Feature Extraction Methods for Paraphrase Detection", Hassan Shahmohammadi, Mir Hossein Dezfoulian and Muharram Mansoorizadeh, ICCKE 2018. [published]
· "Paraphrase detection using Bi-LSTMs and handcrafted features"[in preparation]
___________________________________________________________________________________________
Work Experience
· Working at Parsecoders.com (an Iranian version of freelancer.com) since August 2017 as a programmer.
___________________________________________________________________________________________
Sample Projects
· Artist identification using baseline CNNs and transfer learning (using inception v-3).
· Sentiment analysis using LSTM networks on IMDB dataset.
· Paraphrase detection using varies feature extraction methods such as word2vec, GloVe, pharagraph2vec, TFIDF, etc.
· Short text similarity using convolutional neural networks.
· Generating text using LSTM networks.
· Estimating age from photo using AlexNet.
· Representation of short texts using unsupervised methods.
· Kaggle competition: https://www.kaggle.com/c/dogs-vs-cats
· Kaggle competition: https://www.kaggle.com/c/quora-question-pairs
Github: https://github.com/Hazel1994
_____________________________________________________________________________________
Teacher Assistant
· Theory of Formal Languages & Automata(undergraduate)
· Algorithm design (undergraduate)
· Machine learning( graduate)
_______________________________________________________________________________________
Courses Taken
· Statistical Pattern Recognition, Ref: Introduction to Statistical Pattern Recognition, By Keinosuke Fukunaga
· Advanced Artificial Intelligence, Ref: Learning from data, by Yaser S. Abu-Mostafa
· Natural language processing, Ref: Speech and Language Processing (2nd Edition), by Daniel Jurafsky and James H. Martin
· Machine learning, Ref: Introduction to machine learning (third edition), by Tom Mitchell
· Evolutionary Computing, Ref: An introduction to genetic algorithms, by Melanie Mitchell
· Expert systems, Ref: Expert systems principles and programming (third edition), by Joseph C. Giarratano
· Information retrieval, Ref: an introduction to information retrieval, by manning raghavan and Schutze
· Digital Image Processing, Ref: Digital Image Processing (3rd Edition), by Rafael C. Gonzalez and Richard E. Woods
Professional Skills
· Programming: Python, Java, C#, MATLAB.
Working with python libraries such as NLTK, keras, sklearn, genism, tensorflow, and tflearn.
Working with MATLAB in machine learning domain.
Language Skills
English (fluent)
TOEFL iBT General Score: 109, Listening: 30, writing: 28, speaking: 26, reading: 25.
Persian (native)
Kurdish (mother-tongue)
________________________________________________________________________________________________
References
Hassan Khotanlou, Associate Professor
Department of Computer Engineering, Bu-Ali Sina University of Hamadan.
PhD from Pierre & Marie Curie University, Paris, France.
Email: hkh@basu.ac.ir,
Homepage: http://profs.basu.ac.ir/khotanlou/index.php?L=en&pc
MirHossein Dezfoulian, Assistant Professor
Department of Computer Engineering, Bu-Ali Sina University of Hamadan.
PhD from University of Wollongong, Australia.
Email: dezfoulian@basu.ac.ir,
Homepage: http://profs.basu.ac.ir/h-dezfoulian/index.php?L=en&pc
Muharram Mansoorizadeh, Assistant Professor
Department of Computer Engineering, Bu-Ali Sina University of Hamadan.
PhD from Tarbiat Modares University, Tehran, Iran.
Email: mansoorm@basu.ac.ir,
Homepage: http://profs.basu.ac.ir/mansourizadeh/index.php?L=en&pc