I went back the shopping place which is in front of the Customs office. As suggested by someone on internet, I purchased a local ‘Tashi cell’ SIM from a mall. Just for the sake of getting a Bhutanese cell number as a souvenir , I wanted to purchase the SIM.
Documents required to purchase a local SIM
I have posted some more information on touristhelpline.com
I instantly got the local sim and it activated within few seconds. I called my mother in India to check it. Indian SIMs work upto a certain area of Phuentsholing.
I went back to the transport office. Though I had to move from one office to other, it did not take much time and I got the permit. The employees were gentle and cooperative to me.
After this, I was free to start my journey. Here, I would also like to inform that the first check point, where my entry permits were checked and I had to register myself and bike, was around 10 kms away on Thimphu highway. This is also known as Asian Highway 48 (AH 48).