Post webinar video: Questions & answers

On the webinar, we got a lot of questions that didn’t fit in one video due to limited webinar’s time. Here is an extra video regarding product development, Splynx features, and support.

In this video managing director at Splynx,  Alex Vishnyakov  is talking about

– What is the Splynx development plan for next year?

– How Splynx can help with network design?

– Auto-generation of invoices in Splynx

– How does the field technician interact with a system?

– IPOE, 802.1x supporting

– VOIP plans

– CPE Management

– Export data from Mikrotik to Splynx

– Payments via Mastercard and Visa and other gateways in Splynx

– Equipment and another advice

Case study – Fitel Network S.L.

Fitel Networks is a company from Catalunya, Spain. The company has had its own wireless network for 2 years with over 1,000 subscribers connected.

 

 

 At the end of 2017, the Fitel team started to deploy fibre optics. Fibre was installed in the main urban locations of the city of Lloret de Mar. The topology that the technical management team decided to use is FTTH and FTTB, with GPON technology. Currently, the network is designed to connect over 10,000 fibre ONT devices. Services that are provided to end-users include VOIP and Mobile connections. The company connects companies and residential customers that pay on a monthly basis (recurring) and has a number of customers who come to Lloret de Mar for a vacation and pay for services in a prepaid mode.

 The Fitel Network was optimized by deploying the Splynx ISP billing system, redesigning the core network to allow 10G throughput, registering AS and IPv4 address space and deploying BGP services for their customers.

 

“Many hours of work, some days without sleep, all efforts are rewarded by the core of Fitel Network, which is prepared to compete with the big operators in IP traffic of 10G, implementation of BGP, OSPF, PPPOE servers, routing, implementation of security policies, and also the implementation of the software for ISP. A special thanks to SPLYNX team, Alex Vishnyakov and Paul Gerhardt for their dedication and for believing in this great project.”Erwin Cárdenas Barrera – General Director & CEO of Fitel Network S.L.

 

Let us describe how it started and the whole process

In the middle of 2018, Fitel Networks requested our team to install and deploy Splynx billing software. During installation and implementation, we noticed that the main MikroTik CCR router was having some strange issues. PPPoE users were continuously disconnecting, packets were lost and router behavior was unstable. Fitel’s technical officer Paul Gerhardt asked us to investigate if the problem was caused by the Splynx software, but we knew that Splynx can reconnect PPPoE sessions only in one case – when an administrator sends COA or the ‘kill session’ command. Our engineers were aware of this and started to analyze the network design.

 The company had Huawei MA5608 OLT equipment installed, which worked in bridge mode, terminating all PPPoE connections on a MikroTik CCR1036 router. This router was performing the role of PPPoE concentrator and NAT server. When the network had only wireless customers at around 800 Mbps, the router was performing well, but when the first few hundred GPON customers were connected, the router’s traffic reached 1.4 Gbps and the issues started.

 The main issue was that once a day at around 4pm, several customers reconnected their PPPoE sessions, which caused the CPU activity to jump from 30% to 100%, and, as a result, the router stopped responding to new PPPoE requests. Also, it started to disconnect dozens of already-connected PPPoE customers.

 A short investigation from our team showed that NAT connections tracking together with a large number of PPPoE connections and hence high traffic gives this issue. We offered Fitel Networks our service of network redesign and optimization.

 The initial topology looked as in the picture provided below:

  1. Wireless network. All users connect through the network on L2 bridges up to a CCR router. Customers have UBNT or MikroTik CPE equipment that works as a transparent bridge. Each customer has a TP-Link or MikroTik router installed on the premises which establish a PPPoE connection and also connects end users’ devices to their home Wi-Fi networks.
  2. Fibre network. A Huawei MA5608 router is connected to a MikroTik CCR router with two Gigabit interfaces. Interfaces are bonded using LACP. The company planned to migrate to 10G interfaces and was waiting for the module car
  3. MikroTik CCR1036. The router was connected via a 10G port to the uplink with PPPoE and NAT server running on it.

 

Our team has experience with designing ISP networks for over 10G loads. We have built and optimized several wireless and fibre networks and it is one of the services that a customer can get from us when deploying Splynx ISP software. The main point of the whole optimization was to move out the load from one main MikroTik router and share it between different equipment. There was also another reason – to change the MikroTik router that is CPU routing-based to some hardware accelerated router, for example, Juniper, Alcatel or Cisco.

Fitel Network’s IT team has MikroTik knowledge and, as such, changing the platform was very painless. We decided to take more than one MikroTik router, add switches to have L2 redundant topology, and share the load between different equipment.

The result of the redesign and optimization is presented in the picture below:

Logical topology created in visualization software:

 

Equipment that was installed

A MikroTik CCR1036 router acts as the BGP and NAT router

MikroTik CRS317 16 ports SFP+ – core switch, are connected to uplink 10G ports, a BGP router and all three PPPoE routers are also connected via 10G ports. Please note that one same switch is installed just close to the first one. The second switch acts as a cold backup. This means it has exactly the same configuration as the core switch and the same amount of ports, ready to swap in case of failure of the first switch.

3x MikroTik CCR1036 routers which work as PPPoE servers where fibre and wireless clients are connected.

MikroTik CRS328 24 ports + 4 SFP ports, which acts as a distribution switch. To this switch are connected GPON OLT and links coming from wireless towers. Also, PPPoE router downlinks are plugged-in to this switch.

A BGP router announces public prefixes to the Internet and routes traffic between two Internet providers. Uplinks are 10GB lines. Each PPPoE router has its own private 172.16.x.x network from which we assign IP addresses to private and prepaid customers. It means that on the BGP router there are three installed static routes to these networks – each network to a proper PPPoE router.

PPPoE servers are connected to the same switch on the same VLANs, which means that users from fibre and wireless networks can connect to one of three PPPoE servers at the same time. The advantage of this approach is scalability and failover. If one router fails, customers can automatically reconnect to the two remaining PPPoE routers.

PPPoE routers are linked with Splynx Radius server. When a customer connects to router1, he gets the IP from the pool, which is dedicated for this certain router.

Another story is with public IP addresses. In this case, customers have the static IP address, which means that physically they can connect to any PPPoE server.

To know where the BGP router should send traffic, we have activated OSPF on three links – between each PPPoE concentrator and the BGP router. PPPoE customers IPs are redistributed to OSPF as connected routes as soon as they connect to the Internet. It is important to set the routing filters correctly – allowing only public IPs in a routing table with /32 routes, and all private IPs are not redistributed to the OSPF.

Download case study Fitel Network&Splynx case study

HTTPS/SSL installation

Here in Splynx we are working very intensively on configuration part of the system. Many things that were available for configuration over CLI we have put to the Splynx GUI configuration. One of such examples is HTTLS activation and SSL certificate installation.

You should always protect Splynx server with HTTPS, even if it’s used inside the ISP network. HTTPS and SSL provides critical security and data integrity your server and your user’s personal information.

From Splynx release 2.1,the HTTPS/SSL option can be found under Config -> Main -> HTTPS / SSL

Splynx allows you to import your existing certificate, that was verified by official authority. In this case administrator should upload the SSL private key and Certificate.

In case when you don’t have certificate, you can request the Let’s encrypt free certificate via Splynx configuration. Click on the Let’s Enrypt and save. Important is to enter the domain name or subdomain name that is linked to your Splynx server (point A entry in domain settings to Splynx server IP address)

Now you are two steps far from getting SSL ready and running – click Validate to obtain SSL certificate and then Activate to run the HTTPS. As a result you should see all status messages with green highlighting :

From now, Splynx runs on Secure connection. Let’s Encypt is a trusted free, automated, and open Certificate Authority.


Should you have any questions regarding SSL/HTTPS installation or further information is needed, please contact us or schedule a call with our engineer.

Splynx GDPR compliance

From May 25th 2018, the EU’s General Data Protection Regulations (GDPR) came into effect.

As part of our commitment to transparency, and in preparation for the new data protection laws, we’ve updated our Software solution. We encourage you to read the GDPR Splynx compliancy document in full and to contact us if you have any questions.

Splynx is a management software for Internet Service Providers (referred to as ISP in the following text). It means that our company doesn’t work directly with data of individuals.

However, our clients operate in the B2C market and use our software to store their customers’ data. In this situation, Splynx s.r.o. works as a solution provider. The areas and situations described below are when Splynx software works with the data of individuals and how our software complies with GDPR regulations.

Storing personal information

-> Storing data of individuals in general

Splynx software is installed in the premises of an ISP company. In this case. all data is physically stored in the office or datacenter of our client. This means that our client is in charge of the physical security aspects of the server where the software runs.

Basic information that is stored in a Splynx database about customers is not sensitive. Data such as a customer’s address, phone number, passport/ID number or bank account are requested by ISP based on the customer’s agreement. For example, the ISP cannot provide services without being able to contact the customer and is not able to receive payments if it does not have the customer’s bank details. Technical information regarding a customer’s data tariff plan and a customer’s IP/MAC address is needed by the ISP to provide the service itself. There is no additional confirmation needed by the ISP from the customer to store this data.

-> Customer’s information

The information that is stored by default in Splynx software about a customer is as follows.

ISP administrators can create additional fields and store information about customers in these fields. In the case where these fields feature sensitive data, the ISP company should gain permission from the customer to use this data, Splynx is not responsible for gaining this permission, this is the total responsibility of ISP company.

-> Communication

Splynx software automatically sends emails and SMS to ISP customers in certain situations –  when an invoice is created, when payment is received or when an invoice is overdue and the service is put on hold. These emails are sent to end customers by the ISP because they are needed in order to provide the service and operate its business and as such they  are covered by the main service agreement. No additional confirmation from customers is needed in this instance.

Splynx does not provide a newsletter or similar service. If an ISP wishes to send promotion emails or SMS, they should obtain further confirmation from customers and this confirmation should be stored in the  newsletter/promotions platform.

Splynx stores all emails sent  to customers. Splynx also stores replies from customers in the Tickets section. Tickets can be displayed to customers or can be hidden and available only for administrators.

-> Passwords

Splynx stores all passwords in an SQL database in encrypted format. Any field that is defined as a password is automatically encrypted before saving to the database. The passwords are not displayed in logs or API calls. All customers’ passwords are by default hidden  to administrators. Customers can reset the passwords to their portals using the email reset form.

-> IP addresses

IP addresses and MAC addresses are important for an ISP company to provide access to the internet. IP addresses are stored in the Splynx database in section IPAM – IP address management. IP address and MAC address are unique and can be assigned to one customer at the same time.

-> Logs of sessions

Splynx stores information about IP sessions. The system stores a combination of IP address, the start session date time and the end session date time. This allows an ISP company to find out quickly which customers were using an IP address in a certain period of time.

Administrative access to Splynx software

-> Access to the system

Our software operates with two different access portals. The first is for administrative access and the second is for customer access.

Administrative access allows the administrators of an  ISP company to manage and work with customers’ information. Administrative access can be given by an  ISP to its own employees or to third-parties (for example resellers.) When a third-party receives access to Splynx software, administrative access can be limited and the third-party will only be able to view and work with customers that have a contractual relationship with the third-party.

The customer portal allows customer to login to an ISP system and view information about their services, invoices, payments, payment methods, statistics of internet usage and to  check the durations and cost of telephone calls

-> Access levels, roles and permissions

Access roles  – To achieve more granularity and tighten the security of user accounts, Splynx billing software supports access roles. The software makes it easy for an ISP administrator to assign permissions by providing pre-defined “roles” to choose from. These roles and their permissions are fully customizable at the administrator level.  By selecting one of the pre-defined roles (Super administrator Administrator, Customer creator, Financial manager, Manager, New Role) during user account creation, the billing system applies the relevant permission settings for that role.

Permissions – while the roles are predefined level of permissions, the Permissions themselves define which parts of the system can be displayed to or edited by administrator.

-> Logs of operations

Each single action of administrator in Splynx is tracked. View, Edit, Save, Delete, Rename, Change – anything an administrator does in the system is always saved in logs. All information is located in Administration -> Logs -> Operations. A search by customer, administrator or action is available in this section.

Technology description

-> SSL communication

Our platform is web-based. Administrators and customers access it via web browsers. Preferred communication is HTTPS. Our team helps to setup the SSL communication on ISP

-> Two factor authentication

Splynx software supports two-factor authentication. This is a configuration security feature that enables system administrators to implement customized access to Splinx software tools by setting Google authenticator or similar software.

-> Encryption of passwords

The passwords of administrators, customers, pppoe logins, routers etc are all  encrypted in the SQL database with the server’s key. Passwords are not sent in plaintext in any type of client-server communication (HTTP, HTTPS, API, Exports)

-> Remote backups

Splynx s.r.o. provides a remote backup service to its clients. This means that a client’s server connects to our cloud storage once every 6 hours and pushes the changes from database and application. All data from our clients and their customers is transferred over an OVPN encrypted tunnel. Data is transferred via Burb protocol which is also encrypted. The data is stored in the server in encrypted format. You can read more about our backup platform here – http://burp.grke.org/

Splynx support access

-> Administrative access

We have our own proprietary software solution that is used by our company for license management, backup management and which allows our employees to access client’s servers, provide assistance, resolve issues and restore data in the case of failure.

When the system is installed, our clients can request remote support. Splynx has a tool – splynx-remote-support – and when this tool is installed, the client’s Splynx server establishes an OVPN encrypted tunnel to our cloud platform. This allows us to access the client’s server on a private IP address.

When our employees access the web administration of a client’s Splynx server, they use the  administrator’s name, “splynx-remote-support”. They also use the password that is generated once every day. To access the client’s server, our administrator must also enter a one time code that is created every 30 seconds. This strict authentication allows us to ensure that in the situation where one of  our employees leaves the company and is therefore removed from the authentication system, they will not be able to access a client’s servers.

-> SSH remote access

Similar to administrative access is our SSH access. We connect to a client’s servers via a private IP address of the OVPN tunnel. Our support people connect to the client’s servers with Public Keys and no password. The Key is created based on the private key of the authentication server once every day. The sudo password is also changed once per day.

-> Splynx internal sub-system logs

All access by our employees to a client’s servers is logged. We store information about web access and SSH access. In addition, we also  store the history of commands that are run on the server and web actions are stored in each Splynx server under “Administration -> Logs -> Operations”