Upload Robots txt file in WordPress, now becomes little tricky after the security update, According to this update, you do not have permission to Edit File in WordPress. But thankfully, we have a Cpanel for it. I already cover the importance of WordPress robots.txt file, So now the turn is, how we can upload it.

I this tutorial, I will explain both the methods are

1. By using Plugin
2. Manually upload to Cpanel

But first, here is a little explanation of Robots.txt File for fresher. Whenever search engine crawls your site, It first looks at the robots.txt file and then checks all the command in it and move next. This file tells search engines what they should and should not index from your site. It also may indicate the location of your XML sitemap that helps you to the faster indexing of your website. Maximum bots follow this thumb rule, expected bots that are made for the hacking purpose. You can protect your website by applying protection layer on it.

Upload robots.txt By Using Plugin

You need a plugin called Yoast SEO. This is a free plugin used for SEO purpose in WordPress. You can Alos use All in one SEO Plugin. But I recommended Yoast SEO plugin.

• Login your WordPress as an Admin and click on SEO Tab
• Not click on Tool and then click Check File Editor Option.

Note: –  If File editor option is not available then switch to Method 2

• Now you will see Robots.txt file box with your old Robots file.
• Replace this robots.txt file with your New robots.txt file.

• Now click on save.

After saving your robots file. Check, your file uploaded or not

• Open your browser window and type,

Yourwebsitename.com/robots.txt For example www.hitechwork.com/robots.txt And hit the enter button.

You will see your default robots.txt file are replace with your new file. Upload Robots txt file In WordPress is done.

Upload Robots txt file through Cpanel.

In this case, you need password and username of Cpanel.

• Login your Cpanel and then click on the File manager.

Now you are in the Public_html folder. Where all your content, theme and file are placed.

• Click on New File and hit create new file button.

Now, your blank file for Robots.txt file is ready.

• Click on the file and then click edit on the top of your browser.

• When you click on edit, a new window will be open in editing Mode.

• Copy and paste your robots file here and save and close the window.

Again you have to check your robots file uploaded fine or not. Open your browser window and type the following command and hit the enter button. Yourwebsitename.com/robots.txt Note: – Replace Yourwebsitename.com with your website name. Here is a Short Video Tutorail How to Upload Robots.txt file In WordPress.

https://youtu.be/dnCCKryxzsA

Make Sure Your Robots file Not Blocking Website

Sometimes we accidently block robots to crawl a website. Make sure robots.txt file not stopping search engine to crawl your website (directory of content). You need Google webmaster tool account to ensure that, your robots txt file does not stop search engine to crawl your site.

• Login your webmaster tool Account and click on Crawl.
• Ater that click on Fetch As Google.

• Now copy the URL of any post and paste in the Fetch As Google Box.

Note: – Your domain name already written outside the box, so delete your domain name from the URL Box.

• Now hit the Fetch as Render Button and check the status report.

If your robots file blocks the assessing the page it will show in the report. For example, Look I block my “tag page” from assessing.

See the fetch as google report, shows you,

1. Page block or not.
2. If it blocks, then which file blocking it.

Click on the robots.txt test to check which file blocking the page.

When you click you will see, the file which is blocking your content and highlighted automatically by google. Then make some necessary change in the file or delete it.

Recommended Post

• What To Do After Installing WordPress
• Difference WordPress Categories Vs Tags!

Remember to share this post with anyone who might benefit from this information, including your Facebook friends, Twitter followers and members of your Google+ group! And also Support Us By Liking Our Facebook, Twitter, and Google+ Page.

If you have any suggestion or problem about “Upload Robots txt file in WordPress or Cpanel” please feel free to comment below.