Publication
On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms
dc.contributor.author | Moura, André | |
dc.contributor.author | Lima, Pedro | |
dc.contributor.author | Mendonça, Fábio | |
dc.contributor.author | Mostafa, Sheikh Shanawaz | |
dc.contributor.author | Dias, Fernando Morgado | |
dc.date.accessioned | 2024-02-16T11:41:06Z | |
dc.date.available | 2024-02-16T11:41:06Z | |
dc.date.issued | 2023 | |
dc.description.abstract | Chatbots are becoming increasingly popular and require the ability to interpret natural language to provide clear communication with humans. To achieve this, intent detection is cru cial. However, current applications typically need a significant amount of annotated data, which is time-consuming and expensive to acquire. This article assesses the effectiveness of different text representations for annotating unlabeled dialog data through a pipeline that examines both classical approaches and pre-trained transformer models for word embedding. The resulting embeddings were then used to create sentence embeddings through pooling, followed by dimensionality re duction, before being fed into a clustering algorithm to determine the user’s intents. Therefore, various pooling, dimension reduction, and clustering algorithms were evaluated to determine the most appropriate approach. The evaluation dataset contains a variety of user intents across differ ent domains, with varying intent taxonomies within the same domain. Results demonstrate that transformer-based models perform better text representation than classical approaches. However, combining several clustering algorithms and embeddings from dissimilar origins through ensemble clustering considerably improves the final clustering solution. Additionally, applying the uniform manifold approximation and projection algorithm for dimension reduction can substantially improve performance (up to 20%) while using a much smaller representation. | pt_PT |
dc.description.version | info:eu-repo/semantics/publishedVersion | pt_PT |
dc.identifier.citation | Moura, A.; Lima, P.; Mendonça, F.; Mostafa, S.S.; Dias, F. M. On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms. Appl. Sci. 2023, 13, 5178. https://doi.org/10.3390/app13085178 | pt_PT |
dc.identifier.doi | 10.3390/app13085178 | pt_PT |
dc.identifier.uri | http://hdl.handle.net/10400.13/5558 | |
dc.language.iso | eng | pt_PT |
dc.peerreviewed | yes | pt_PT |
dc.publisher | MDPI | pt_PT |
dc.relation | Laboratory of Robotics and Engineering Systems | |
dc.relation | Laboratory for Robotics and Engineering Systems | |
dc.rights.uri | http://creativecommons.org/licenses/by/4.0/ | pt_PT |
dc.subject | BERT | pt_PT |
dc.subject | Chatbots | pt_PT |
dc.subject | Embedding clustering | pt_PT |
dc.subject | Intent detection | pt_PT |
dc.subject | Natural language processing | pt_PT |
dc.subject | Natural language understanding | pt_PT |
dc.subject | RoBERTa | pt_PT |
dc.subject | Word and sentence embedding | pt_PT |
dc.subject | . | pt_PT |
dc.subject | Faculdade de Ciências Exatas e da Engenharia | pt_PT |
dc.title | On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms | pt_PT |
dc.type | journal article | |
dspace.entity.type | Publication | |
oaire.awardTitle | Laboratory of Robotics and Engineering Systems | |
oaire.awardTitle | Laboratory for Robotics and Engineering Systems | |
oaire.awardURI | info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F50009%2F2020/PT | |
oaire.awardURI | info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UID%2FEEA%2F50009%2F2019/PT | |
oaire.citation.issue | 8 | pt_PT |
oaire.citation.startPage | 5178 | pt_PT |
oaire.citation.title | Applied Sciences | pt_PT |
oaire.citation.volume | 13 | pt_PT |
oaire.fundingStream | 6817 - DCRRNI ID | |
oaire.fundingStream | 6817 - DCRRNI ID | |
person.familyName | Silva Mendonça | |
person.familyName | Mostafa | |
person.familyName | Morgado-Dias | |
person.givenName | Fábio Rúben | |
person.givenName | Sheikh Shanawaz | |
person.givenName | Fernando | |
person.identifier | 34497 | |
person.identifier.ciencia-id | 7F1E-8AE9-3098 | |
person.identifier.ciencia-id | EE14-BEB3-F82B | |
person.identifier.ciencia-id | 7B14-DF07-AA6D | |
person.identifier.orcid | 0000-0002-5107-3248 | |
person.identifier.orcid | 0000-0002-7677-0971 | |
person.identifier.orcid | 0000-0001-7334-3993 | |
person.identifier.rid | N-9228-2015 | |
person.identifier.scopus-author-id | 55489640900 | |
project.funder.identifier | http://doi.org/10.13039/501100001871 | |
project.funder.identifier | http://doi.org/10.13039/501100001871 | |
project.funder.name | Fundação para a Ciência e a Tecnologia | |
project.funder.name | Fundação para a Ciência e a Tecnologia | |
rcaap.rights | openAccess | pt_PT |
rcaap.type | article | pt_PT |
relation.isAuthorOfPublication | 0dbad4bf-3a0c-4c43-8022-42199a5e09c0 | |
relation.isAuthorOfPublication | f90aafd0-eedb-47ea-945a-40b1c1fe802a | |
relation.isAuthorOfPublication | 042f7593-c6ca-4553-8f0e-12ccf17018be | |
relation.isAuthorOfPublication.latestForDiscovery | 0dbad4bf-3a0c-4c43-8022-42199a5e09c0 | |
relation.isProjectOfPublication | c0352e81-99e6-4923-99ed-074df09e4db0 | |
relation.isProjectOfPublication | eda3c844-853e-4c97-83da-99235180d98c | |
relation.isProjectOfPublication.latestForDiscovery | eda3c844-853e-4c97-83da-99235180d98c |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms.pdf
- Size:
- 2.1 MB
- Format:
- Adobe Portable Document Format
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 1.71 KB
- Format:
- Item-specific license agreed upon to submission
- Description: