ahiddenmessi2 OP t1_jad9upi wrote on February 28, 2023 at 4:41 PM Reply to comment by I_will_delete_myself in [D] Training transformer on RTX2060 by ahiddenmessi2 Thank you. I am looking at codeBERT which might satisfy my needs Permalink Parent 1
ahiddenmessi2 OP t1_jaciwqg wrote on February 28, 2023 at 1:31 PM Reply to comment by ggf31416 in [D] Training transformer on RTX2060 by ahiddenmessi2 Thanks for your reply. My goal is to train the transformer to read a specific programming language so I I guess there is no pre trained model available. Seems I have to train it from scratch on my laptop GPU :( Edit: and yes it has 6gb only Permalink Parent 1
ahiddenmessi2 OP t1_jacin2n wrote on February 28, 2023 at 1:29 PM Reply to comment by KingsmanVince in [D] Training transformer on RTX2060 by ahiddenmessi2 My dataset size can be varied cuz the data can be generated. Also, I will consider using gradient accumulation to improve performance too. Thanks Permalink Parent 1
ahiddenmessi2 OP t1_jachs8r wrote on February 28, 2023 at 1:22 PM Reply to comment by aigoritma-1 in [D] Training transformer on RTX2060 by ahiddenmessi2 Thank you I will look into it Permalink Parent 1
ahiddenmessi2 OP t1_jacahig wrote on February 28, 2023 at 12:10 PM Reply to comment by CKtalon in [D] Training transformer on RTX2060 by ahiddenmessi2 Thank you . I will take a look of my number of parameters . Permalink Parent 1
[D] Training transformer on RTX2060 Submitted by ahiddenmessi2 t3_11dzfvf on February 28, 2023 at 6:55 AM in MachineLearning 11 comments 0
ahiddenmessi2 OP t1_jad9upi wrote
Reply to comment by I_will_delete_myself in [D] Training transformer on RTX2060 by ahiddenmessi2
Thank you. I am looking at codeBERT which might satisfy my needs