Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
6-2026
Abstract
In this work, we develop a novel reasoning approach to enhance the performance of large language models (LLMs) in future occupation prediction. In this approach, a reason generator first derives a “reason” for a user using his/her past education and career history. The reason summarizes the user’s preference and is used as the input of an occupation predictor to recommend the user’s next occupation. This two-step occupation prediction approach is, however, non-trivial as LLMs are not aligned with career paths or the unobserved reasons behind each occupation decision. We therefore propose to fine-tune LLMs improving their reasoning and occupation prediction performance. We first derive high-quality oracle reasons, as measured by factuality, coherence and utility criteria, using a LLM-as-a-Judge. These oracle reasons are then used to fine-tune small LLMs to perform reason generation and next occupation prediction. Our extensive experiments show that: (a) our approach effectively enhances LLM’s accuracy in next occupation prediction making them comparable to fully supervised methods and outperforming unsupervised methods; (b) a single LLM fine-tuned to perform reason generation and occupation prediction outperforms two LLMs fine-tuned to perform the tasks separately; and (c) the next occupation prediction accuracy depends on the quality of generated reasons. Our code is available at https://github.com/Sarasarahhhhh/job_prediction.
Discipline
Artificial Intelligence and Robotics | Databases and Information Systems
Research Areas
Data Science and Engineering
Publication
Proceedings of the 30th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2026, Hong Kong, China, June 9-12
First Page
312
Last Page
324
ISBN
9789819214679
Identifier
10.1007/978-981-92-1468-6_20
Publisher
Springer
City or Country
Cham
Citation
DONG, Shan; ACHANANUPARP, Palakorn; MAI, Hieu Hien; WANG, Lei; LU, Yao; and LIM, Ee-peng.
On reasoning behind next occupation recommendation. (2026). Proceedings of the 30th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2026, Hong Kong, China, June 9-12. 312-324.
Available at: https://ink.library.smu.edu.sg/sis_research/11378
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1007/978-981-92-1468-6_20