From ISP admin

Shaping and MikroTik router options

This document depicts the fundamental information needed for correct understanding of the cooperation of the ISP admin system with the MikroTikrouters. Without the following knowledge, you will not be able to fully and correctly define important setting in the ISP admin.

Updating the configuration of all routers is run periodically every 5 minutes. If you will, however, debug one of the routers, you have the possibility of updating only its configuration at any time ( duplicating the rules ) by the button of Router configuration udpates in its header.

The configuration is transferred "distantly". It means that the system during the configuration update firtsly dowloads the current router configuration , performs the comparing of the configuration which should be in the router, and then trasnfers the detected differences. Thus the number of records to flash routerboards is minimized, by which the record is being minimized and thus also occurrence of defective blocks.

Router monitoring is done by means of the SNMP protocol and/or by the API in case it is activated. All changes in configuration which the ISP admin transfers to the router, are transferred through the SSH protocol, or again by means of the API interface. The condition is correct funtioning of the router, ie. the active communication between the router and the ISP admin system.

• SNMP - Ensures router monitoring (traffic, data transmitted, CPU, RAM, etc... ).
• SSH - Duplicating the rules, automatic configuration backups (text as well as binary ).
• API - Moghtier communication with the router, reading out of rOS version, WIFI signal graphs ( SIGNAL, NOISE, SNR, TCQ, etc... ).

Verification of the router connection may be done by the button in its header Test connection. To verify the connection is possible to be carrued out also in a mass way on all routers by the same button at the right top part of the bookmark of Routers / All.

Add the routers first on the monitoring mode, it means that the given routers are seen by the system and they are displayed in the administration, but no change in configuration is now written in it. In case the router is in the "monitoring mode", the system only passively reads out the information by means of the SNMP and/or theAPI and in five minute intervals various values are drawn, as well as statistics and graphs.

In case the connection on router is OK, you can switch the status of the router to active, thus you allow duplicating changes in configuration by means of the SSH, or the API.

Duplicating the configuration on the router

The ISP admin completly duplicates necessary rules into the MikroTik router configuration. The system in all cases works only with those rules which have "ispadmin_" ( Firewall, QueueTree, ppp ) listed in the comment, or have "ispadmin_" ( Mangle ) listed in the packet mark.

Thanks to this solution, it is possible to add own Firewall rules, Queue Tree restrictions, etc., to the router, and the rules defined by you will remain intouched. These rules remain at the beginning of Firewall, Mangle, Queue Tree and the ISP admin never modifies them. The rules will be of course carried out in the given order.

In case of a manual change of any rule ( eg. throug the Winbox ), which is duplicated by the ISP admin system, this manually edited rule will be rewritten by the previous rule set in the system during next configuration update. All changes of settings of the speed of the client, tariff, etc. must be done in the ISP admin.

The ISP admin system thus modifies only the following parts of the configuration and does not interfere into other parts of the ROS:

/ip firewall filter

Here, the firewall rules are recorded. The IP address of the clients will be allowed first, then their end devices, routers which are in another given segments, and in the end, the whole sub-network is banned in the FORWARD by the rule of DROP. To be able to add a client into the system at all, the whole sub-network must be entered as the "IP address for the WIFI users" in the bookmark of Routers in the item of Routed netowrks.

Firewall duplicating may be switched off in the router settings by "clicking" off the item of Apply the firewall rules. If this item is not checked off, the firewall rules will not duplicate on the router. This may be used for instance in case you do not hacve all the users in the system. In this case the firewall would block the users and it is thus better not to apply firewall.

Normally, IP addresses restrictions (SRC as well as DST) are duplicated into the firewall. In some cases ( eg. for the reason of lowering the number of the rules in the firewall ), it is necessary to restrict only to SRC and/or DST. This type of duplicating firewall may be set in the menu ofSettings / System settings / MikroTik. The type of duplicating of the rules will be set in the item of mikrotik_fw_drop.

/ip firewall mangle

Here, packet mangling takes place, when the client receives the "packet mark" in the form of the "ispadmin_ID", where the ID is the unique number of the client in the database. In case the client has more IP addresses and/or has other "IP ranges" set in the Internet service settings, all these IP ranges have the same packet mark and thus are restricter in the same Queue Tree.

/queue tree

Here, the Queue Tree is set. The options for setting Queue Tree are relatively extensive and are described below.

/ip firewall nat

Every client has their Internet service suspended, they are recorded in NAT, where redirecting of its processes to the "Information page" is ensured, which may be edited in the menu of Settings / Info page. Own suspension is done in the Client card in the Internet service. The "Reason for suspending" may be also selected here and according to this option, displaying page will be selected.

The requirement for the rules duplicating into NAT is the key setup "mikrotik_nat_disable_users" onto the value "1". The settings is done in the menu of Settings / System settings / MikroTik. If the value here is "0", the rules are not duplicated into the NAT. This may be used eg. in case redirecting of the suspended users is solved in any other way outside of the ISP admin.

/ppp secret

If you use PPPoE, and in the Internet service in the client card "Type of user" is selected PPPoE, you can fill in the name and password of the client. This name and password wil be duplicated into the table including the client IP address.

Default firewall

In the menu of Settings / System settings / MikroTik it is possible in the item of "mikrotik_default_fw" to define default firewall rules which will be duplicated on all routers where the access through SSH is allowed. Thus it is for instance possible to restrict sharing of the OS Windows, spreading of specific viruses by blocking particular ports.

The rules which are lisre in the default firewall must always begin with "/ip firewall filter add". The rules which are not defined in this way, will be ignored.

Example of default firewall:

/ip firewall filter add chain=forward protocol=tcp dst-port=135-139 action=drop comment="netbios"
/ip firewall filter add chain=forward protocol=udp dst-port=135-139 action=drop comment="netbios"
/ip firewall filter add chain=forward protocol=tcp dst-port=445 action=drop comment="netbios"
/ip firewall filter add chain=forward protocol=udp dst-port=445 action=drop comment="netbios"
/ip firewall filter add chain=forward src-address=10.0.0.0/8 protocol=udp dst-port=53 action=accept comment="DNS" 
/ip firewall filter add chain=forward src-address=10.0.0.0/8 protocol=icmp action=reject reject-with=icmp-admin-prohibited

MAC filter

On every router you can activate the MAC filter. The activation is done for the router by checking off the item of MAC filter. If the MAC filter is active, the IP as well as the MAC address will be duplicated into the firewall. If the client changes their IP or MAD addresses, the Internet will not work for them.

The system automatically reads out the MAC addresses from the routers once an hour. In case you then have the MAC filteractivated and you are adding a new client, you do not have to fill in the client`s MAC address. The system in this case ( if the MAC is not filled in ) allows the client on the firewall and when the client connects, the system will upload the MAC address from the ARP tabel on the router and it will assign it to the client. Then the IP and MAC address relation will be "locked off" for the client.

In case of necessity in some cases to switch off the automatic loading of the MAC addresses of a certain client, then instead of the MAC address, fill in "-" ( dash ). Thus the ISP admin will be allowed on firewall and therefore the MAC lock-off will not be performed. This will be used in case when for example two computers take turns at one IP address ( laptops, PDA, etc. ).

Automatic loading of the MAC address is thus makes it possible for all users to switch the system variable "macfilter_getusermac" in the global settings in the menu of Settings / System settings / Public.

Queue Tree and duplicating into the router configuration

Generally

The ISP admin system will carry out the record of the rules into the Queue Tree according to the settings. Simple Queue is not supported because it does not allow setting of such parameters as the Queue Tree. The system "multiplies" the set speeds by the value of rate_exponent in the set tariff before duplicating into the Queue Tree. This value may be set in the menu of Settings / System Settings / Public. The value of rate_exponent indicates the speed which the client actually reaches. rate_exponent at 1.2 is set as the default value so at the speed of 1024 kbit/s the speed of 1228 kbit/s is duplicated into the Queue Tree. By this increase in the tariff speed, the protocol control is basically "covered" and potential packet loss which may appear on WiFi. The customer is satisfied because they usually get a higher speed than which is stated in the contract. Sometimes it is difficult to explain the customer that if their speed is 1024 kbit/s, they will never see the download speed of 128 kB/sin his browser due to the protocol control.

In the menu of Settings / Tariffs you can set all the parametres of the tariff such as the speed, type of line, burst, aggragation, etc...

"Global-in" vs. "global-out" setup

For every router you can select a popup menu of the "Queue type", this settings influences how the Queue Tree will be processed. The options are "global-in" or "global-out". This settings is essential for correct functioning of the router. The whole set of packets must be studied for better understanding according to the figure:

If the QOS is carried out on the central router, this selection is inactive because MANGLE is governed by the cetnral router settings. If the change is done in the router settings ( eg. from global-in to global-out ), firstly the Queue Tree and MANGLE are completely removed and re-set again according to the new settings. This happens because of "purification" of all Mikrotiku tables so that the rules are not mistakenly duplicated.

Individual options:

global-out

• Default settings of the router.
• The packets directed from the client ( dst-address ) are marked in MANGLE in POSTOUTING.
• The packets directed towards the client ( src-address ) are marked in MANGLE in PREROUTING.
• This settings is used in case the NAT is used on the router, as the packets are correctly marked and correct client addresses are seen in POSTROUTING and not their preNATed public addresses.
• The disadvantage is that every packet must "go through" the whole firewall, which increases the router`s load. However, in case also NAT is run on the router, this is the only way how to apply QOS and data flow restriction on the client.

global-in

• The packets directed towards the client ( src-address ) and from the client ( dst-address ) are marked in MANGLE in PREROUTING.
• Using global-in lowers the router`s load.
• This option cannot be used in case that NAT runs on the router. In this case, dowload would not be restricted for the client because the packets emter Firewall being NATed ( under public address ) and thus are not correctly marked, and therefore will not assigne to the correct Queue Tree.

Local and central shaping

For every router ( the "Routers" bookmark) it is possible to select a router on which shaping takes place from the popup menu of "QOS run on the router" for the Mikrotik routers. If the option "Locally" is selected, QOS is performed directly on this router. When selecting another router from the menu, shaping of clients is performed on the target router. If shaping is set on another router ( and not locally ), the router header will show a note that the clients on this router are being shaped on another router. When adding or editing the Internet service, in case of shaping of clients on another router, a note will appear stating on which router shaping is being performed.

Local shaping has an advantage in the fact that the client is restricted yet at the closest access point and the totalload will be lowered because only a few clients are restricted on every access point. The disadvantage, however, is if we use aggregated tariffs, eg. if we have 3 clients on the access point with the aggregated tariff 1:10, then it is technically impossible at this number of clients to ensure the aggregation. In this case it is better to ise central shaping which shapes clients on one router from eg. 5 access points ( eg. within the town or city ). Thus more clients "meet" on one router and aggregation does not apply. The disadvantage of central shaping lies in the fact that more efficient hardware must be used, the one which is able to shape larger amount of client.

Queue-type setting

On the Mikrotik router you can set the system of "data flow control", which is important for the right functioning and correct ensuring of the connection quality. The settings is done in the menu of "Queues / Queue type".

Types of Queues are:

PFIFO - Packet First In First Out

PFIFO and BFIFO sends the packets as they come. It means that the packet which came first, leaves first. This setting of the "QueueType" is not suitable for shaping because a meaningful shaping cannot be ensured at such principle. The line becomes "congested”, the responses are high.... This type of QT is set as default and it is necessary to change it.

SFQ - Stochastic Fair Queueing

This is the only simple algorithm to ensure equal network utilization by all active dated flows. It does not ensure, however, fair utilization among the users or programs, but eg. among individual TCP connections. It creates more queues into which individual TCP connections and UDP flows are divided according to the hash function. These queues are processed by the "round robin" algorithm. Every connection thus has a chance to be effected and one big data flow will not congest the most of the line`s capacity. The hash function changes in time and thus collisions are solved (two flows in the same queue will not stay for a long time).

RED - Random Early Detection

Is designated for heavy spine connections. According to calculated statistics it reduces the load by throwing away the packets to stop congesting of the line. The closer the total traffic is to the maximum value, the more are the packets thrown away. This algorithm does not provide any options for shaping and treats all the data flows in the same way. On overloaded devices, this however, does not have such high computing demands and keeps clearness as large as possible and the line delay as lowest as possible.

PCQ - Per Connection Queue

Makes restricting of a larger number of unshared line easier.

FULL DUPLEX ( FD ) Tariff

In this case only a simple QT is set, here upload and download are restricted. In case of thie tariff ( eg. 1024/512 ), the client may download 1024 kbit/s as well as send in the speed of 512 kbit/s ( in total 1,5 Mbit/s ).

HALF DUPLEX ( HD ) Tariff

When using this tariff, the upload and download are restricted. At the same time, bothe these QTs will be input under a superordinate QT, which ensures the client may independently download 1024 kb/s, send 512 kb/s, but if they run donwloading and sending at once, the total of these speeds will not exceed 1024 kb/s. The advantage of this is that the customer may run the test which checks download and upload separately, so everything is OK. If, however, the client downloads and at once some torrent which sends, the speed will be no faster than 1024 kb/s.

For setting this type of line is download, in this case ispadmin_7837_down on value "0". This is OK, because the speed is limited by the superordinate "queue ispadmin__7837". Thus the CPU of the router is spared because the QT is only limited once in the superordinate QT. The speed of the superordinate QT is identical with the speed of the download, therefore it is not necessary to limit it separately. If the line had the speed of 1024/1024, the speed would be set to "0" also for the QT ispadmin_7837_down. QT technically are not necessary, but can be ( together with the zero speed ) recorded due to the data counting, graphs drawing and the statistics.

Shared( aggregated ) HD line

When using this aggregated line, a superdordinate line will be set up which will be calculated from the theorem:

( download * number of clients ) / aggregation = total aggregated line

A practical example - in case of the aggregated line 1:10 at the speed of 1024/512 at the number of 20 clients, the total line will be calculated as following:

( 1024 * 20 ) / 10 = 2Mbit/s

A line 2 Mbit/s will thus be created on the MikroTik and all clients with this aggregation will be assigned to this line. The total „aggregation“ groups is dynamically set according to the current number of clients. So if the aggregation group contains 40 clients, the total aggregation group will have the speed of 4Mbit/s. Each client who is assigned to this aggregation group is limited by maximum speed of 1024/512, so no client will have the speed higher than 1024/512 when we sum the upload and download ( in the same way as with the classic HD tariff) and at the same time all these clients together do not exceed the speed of 2Mbit ( again upload and download together ).

In our example, QTs of a shared HD tariff are dislayed. We are speaking about the tariff of 1024/512 kb/s with the aggregation of 1:2. According to the extent of the mentioned theorem, a "superordinate" line is calculated - "shared_629". 3 users are assigned to this line and these are limited by the speed of 1024/512 and this download + upload is moreover totally limited for every client by the speed of 1024 kbit, which limits the sum of download + upload ( eg. Test_user1__7837 ).

Shared FD line

If the FD line is aggregated, download and upload will be separated in the aggregation group. It means that the calculation of speed is done for every direction separately. So in case of the aggregated FD line 1024/512 with the aggregation of 1:10 and the number of 20 clients, the following calculation will be carried out for donwload:

download = ( 1024 * 20 ) / 10 = 2Mbit/s upload = ( 512 * 20 ) / 10 = 1Mbit/s

An aggregation group will be thus created for download (2Mbit), and another one will be created for upload ( 1Mbit ). Download and upload of all clients in this aggregation group will be listed in these two groups.

Here a shared HD tariff of 1024/512 kb/s with the aggregation of 1:2 is displayed. In this case, the speed for download is calculated separately ( shared_629_down ) as well as a separate one for upload ( shared_629_up ). Upload and download of all clients with this tariff will be listed in these two lines separately.

Assigning the clients to a special Queue Tree

In some cases, it is neccessary to assign the client into a special QT, eg. to divide the processes on the router into 3 groups where every one of them represents one connection eg. to another village. Each of these connections has a defined capacity ( Eg. 2 Mbit ).

In this case, there is the need for the Mikrotik to have one Special QT created for every group and set appropriate speed to them, or set manually some other parameters of this QT because every provider has certain requirements or it is needed to create more deep embedded QT structures which cannot be created automatically.

It is neccessary to create named QTs ( which will be implemented into the system ). If the QT is named eg. VIP, then on the Mikrotik there has to be created:

VIP – into this QT, the clients with the HD tariff will be assigned ( upload and download are shared )

VIP_IN – into this QT, client`s download will be assigned here (if the FD tariff is set)

VIP_OUT – into this QT, client`s upload will be assigned here (if the FD tariff is set)

in the example two groups are created for the clients, USERS and VIP. For every group, speed parametres are set according to the need where the total speed of the group will be set as well as the speed for download and upload, or it is also possible to allocate other restrictions such as for example restriction of P2P networks or the number of connections...

The whole tree may be created according to the need. The system "is interested" only in the "end branches" of the whole tree where the clients will be assigned. In our case for the VIP it is VIP, VIP_IN and VIP_OUT. It does not matter in what level the tree end "branch" is.

Now it is necessary to let the ISP admin system „know“ that a special QT VIP exists. This will be set on the router ( In the Routers bookmark ) under the bookmark of the „Queue tree“. Here you need to add a special QT „VIP“. QT VIP_IN and VIP_OUT do not have to be entered. The existence of these QTs is presupposed already when enetering the QT VIP. If the QT VIP_IN and VIP_OUT are not saved on the Mikrotik, the clients, who have the FD tariff set, will be assigned to the QT of "global-out" outside this "tree".

Upon clicking in the on the icon of „Test connection“ for the router, the test of QT will be carried out, which should run correctly.

If the QTs are correctly set on the router and the „test“ runs correctly, then for every client there can be setting for assigning them to this special QT. This is possible when adding or editing the Internet service in the client card. When editing or adding the Internet service by selecting the set tariff ( provided the aggregation is 1:1), another popup menu of „Queue tree“ appears, where QT may be selected into which the client will be assigned. QT selection is possible only when the aggregation is 1:1, otherwise the selection of the special QT WILL NOT APPEAR. That is because of the fact that if the client is assigned in an aggregated (shared) tariff, they cannot be assigned to a special QT because the client cannot belong to an aggregated line as well as to be assigned in a special QT. It is technically not possible to mangle the client packet twice. A packet may only have one packet mark and thus can logically belong only to one line ( aggregated or shared ).

So if you assign a Test_user1 client into the QT VIP, they will be assigned on the MIkrotik as following:

All clients have the tariff 1024/512 with the aggregation 1:1 and it is the HD tariff. The Test_user1 client and the Test_user2 client are assigned to the QT VIP, the test_user3 client is standardly assigned to the "global-out" because they do not have any QT set. As this is the FD tariff, the download and upload of the clients will be divided into the VIP_IN and VIP_OUT

If the tariff is set on HD, the clients will be assigned as follows:

The clients Test_user1 and Test_user2 are assigned into the QT VIP and not into the VIP_IN and VIP_OUT like it is for the HD tariff. All clients have set the classic HD tariff, where download and upload are limited by the superordinateQT ( Eg Test_user1__7837 ). The client Test_user3 is assigned to the "global-out".

Above mentioned methods and processes may be combined according to the need. In our example, the clients Test_user1 and Test_user2 have set the FD tariff with the aggregation of 1:1 and speed of 1024/512. The client Test_user3 has set the HD tariff 256/256, therefore they are assigned to the QT VIP.

preffered_queue in some cases it is necessary to assign the client to the QT manually. For example, in case we need to set the exact total speed of the line. This is possible to be set in the previous method of assigning the client into a specific QT. This is, however, rather effort-demanding and you can easily make mistake by not assigning a client to the QT and then they will automatically become assigned to the global-out. For this reason, you can set the QT in the menu of „Settings / System settings / Mikrotik“ by using the key of preffered_queue, into which all clients will be assigned in case they are not explicitly assigned by the previous method into another QT. It again holds that if we do set preffered_queue "VIP", there has to be the QT VIP, VIP_IN, VIP_OUT created on the router.

Control from the command line

Configurations update is run every 5 minutes. If there has no change appeared on the router, such as change in the client`s tariff, change in the router settings, etc., configuration update is not run on this server. In case that the configuration update is running, it is transferred only on changes and it does not delete the configuration completely with its consequent reload. The system "downloads" the current rules from the MikroTik configuratoin and compares them with its own configuration which should be in the router and the changes will be transferred on the router. Thus, the number of records on the flash router is downsized.

If there is the need to run the configuration update immediately ( not within the 5 minutes ), you can do so from the command line by launching the command of:

/usr/local/script/ispadmin/update_conf.pl

PIf we launch the command for configuration update without a parameter, a help will appear with the list of routers including their IDs:

Automatic mode according to the system status:

/usr/local/script/ispadmin/update_conf.pl auto 

Generates new docsis files for all modems:

/usr/local/script/ispadmin/update_conf.pl gendocsis

Generates new docsis files for router with the given ID:

/usr/local/script/ispadmin/update_conf.pl gendocsis <router ID>

Configuration upload onto all routers:

/usr/local/script/ispadmin/update_conf.pl reload all 

Configuration upload on the router with the given ID:

/usr/local/script/ispadmin/update_conf.pl reload <router ID>

"Hard" configuration upload on the router with the given ID:

/usr/local/script/ispadmin/update_conf.pl reload <router ID> force

Available routers:

ID		 IP			 status		status in DHCP	Router name
-------------------------------------------------------------------------------------

 91 		 88.103.224.215	         online	 	stopped	 	home
101		 10.0.0.1		 online	 	stopped		 mk1

If you wish to update router number 30 ( internal unique number of the router ), launch the command:

/usr/local/script/ispadmin/update_conf.pl reload 91

The result looks as following:

16.2. 2009 13:24:30  system	 Updating the router 91, IP 88.146.172.90, home
16.2. 2009 13:24:30  home	 --- Setting the router home, IP 88.146.172.90, ID 91
16.2. 2009 13:24:30  home	 --- Checking the new network interafaces
16.2. 2009 13:24:30  home	 --- Downloading data from the router
16.2. 2009 13:24:32  home	 --- Setting client restrictions
16.2. 2009 13:24:32  home	 --- Updating router configuration  1/2 ( 3360 b - rows 10 )
16.2. 2009 13:24:32  home	 --- Updating router configuration 2/2 ( 360 b - rows 2 )
16.2. 2009 13:24:32  home	 --- Checking the QTREE dependency
16.2. 2009 13:24:33  home	 --- Updating the routeru 91, IP 88.146.172.90, home finished