SEO spider

© 2003-2015 Denis Suro
https://www.iqlevsha.ru

Free and open source software under the GNU GPL

SpiderSEO is a script with a graphical UI that automatically generates meta tags from the contents of the pages of the website. The meta "keywords" and "description" are taken from the contents of the page. Additional meth are generated as an option.
The script runs on the local image of the site, not online, but can be adapted.
Note that SEO (Search Engine Optimization) is a technique for improving pages for positioning by search engines.



Introduction

The clearer your code is, the better the program will work. For what I call clear and confusing code, see the examples below:
 <head>
 <meta name="keywords" content="ordre, clarté">
 </head>
 
 <head > <
 meta surnom
 = "keyword" content = confusion désordre
 > < /head>
 

In the event that your code is also confused, the resulting code generated by SpiderSEO may not be what you expect.

The SpiderSEO script is planned to be run on the local image of your site (if your computer is your own server, then the local image is the site itself).
Before using SpiderSEO, make a careful copy of the entire directory containing the site files.

Screen

A large text box displays reports and changes. The buttons and other elements are described below .

Create and present reports using the GUI

In the first field, enter the full path to the home page of the site, for example:

  c:\site\index.html
Then click the Generate button to start the generation process.
Changes are displayed in a large text field.

To find out the current meta tags and links on the pages, click the Report button.

File menu

- The Browse command allows you to find and select the page to parse.
- The New List command opens a dialog box for creating a list of pages to process. See below.
- Save - if you want to save the results displayed in the file.
- Exit: Terminates the program.

Create a list

The graphical interface allows you to create a list of links in a file, and this file can be used as the main page for assigning pages for processing.

The add button adds the page to the list.
The delete button removes the page from the list.
The processing order can be changed with the up and down buttons.

Once the list is created, it is saved with the save button.
You can download the list using the download button to add more links.
The new button erases the list.

Click rollback to close the window.

After creating a list in a file, the file name must be entered in the first field as the main page for processing, and the depth must be assigned at least 1.

Help Menu

- Manual: displays brief help.
- A: program links...

GUI Options

Generation parameters

Report parameters

Use the command line utility


The GUI actually calls programs that can be used directly.
If your site is stored in the c :\site path, if the home page is index.html (it can be index.php, etc.), just type:

spider c:\site\index.html ...pour générer les méta tags.
metarep c:\site\index.html ...pour afficher les métas et les liens.

Scripting Options

Recursion
With this option, you can limit the recursion level:
-r followed by recursion level, e.g. 5 (default 0).
spider -r5 c:\site\index.html


To strain
-f replaces existing meta tags.
The generation or replacement algorithm is specified as the header of the spider.sol source file.

Check first
-v allows you to view results without editing files.

Indication

-q without display.

Select Pages to Edit with the Layout Designer

Makelist is a script that automatically creates a list of web pages inside a directory. Once created, you can edit the list to select pages...

  1. Enter:
        makelist répertoire nomliste.html
    
    - the directory contains an image of the site.
    - nomliste.html - the name of the file that will contain the list of pages.
  2. Edit the list to delete files that cannot be edited.
  3. Use the file as the home page of your site.
    Command:
        spider nomliste.html
    
    or to prevent recursion beyond references in the list:
        spider -r1 nomliste.html
    
    The - r1 flag gives a recursion depth of 1. The list itself is recursion level 0.

Non-English languages

To use SpiderSEO with a language other than English, you must replace the list of words to exclude in small.en with an equivalent in that language.
small.xx file, such as small.fr, can be easily created using dictmake, a set of available iqlevsha.ru scripts (see below).

Loading

Resources