标签: Security

  • Suricata学习

    Suricata是一款开源免费的网络威胁检测系统,可以在网络中作为IDS(Intrusion Detection System,入侵检测系统)、IPS(Intrusion Prevention System,入侵防御系统)和NSM(Network Security Monitoring,网络安全监控)使用,同样还可以离线分析pcap文件。Suricata使用专门的语言编写的规则来对网络流量进行分析,还可以利用Lua脚本来更加精确地分析,并以类似YAML或JSON的形式输出,可以方便存储在数据库中。目前Suricata项目属于OISF所有,OISF是一个非营利组织。

    Installation

    Suricata和其他的Linux软件类似,也具有两种安装方法,即直接安装发行版和编译安装。

    PPA安装法

    以下操作均在Ubuntu 16.04中,其他发行版可以查阅官方wiki。

    sudo add-apt-repository ppa:oisf/suricata-stable
    sudo apt-get update 
    sudo apt-get install suricata 
    

    编译安装法

    首先需要安装程序依赖库

    sudo apt-get -y install libpcre3 libpcre3-dbg libpcre3-dev \
    build-essential autoconf automake libtool libpcap-dev libnet1-dev \
    libyaml-0-2 libyaml-dev zlib1g zlib1g-dev libcap-ng-dev libcap-ng0 \
    make libmagic-dev libjansson-dev libjansson4 pkg-config
    

    下载源码

    VER=3.1
    wget "http://www.openinfosecfoundation.org/download/suricata-$VER.tar.gz" 
    tar -xvzf "suricata-$VER.tar.gz" 
    cd "suricata-$VER" 
    

    配置安装

    ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var
    make
    sudo make install
    sudo ldconfig
    

    Suricata还提供了一些自动安装的脚本

    make install-conf 自动创建和安装配置文件
    make install-rules 自动从Emergeing Threats下载最新的规则集
    make install-full 将上面两者都包括
    

    Setup

    接下来我们需要部署Suricata,保证下面的命令均具备管理员权限。

    mkdir /var/log/suricata # 日志信息
    mkdir /etc/suricata # 配置文件
    cp classification.config /etc/suricata
    cp reference.config /etc/suricata
    cp suricata.yaml /etc/suricata
    

    /etc/suricata/suricata.yaml文件中正确配置好变量。HOME_NET设置为本地网络的IP地址,而EXTERNAL_NET建议的设置值是!$HOME_SET,这样所有不是本地IP的流量均被当作外界地址,当然设置成any也是可以的,只是这样的话会产生一些假的警报。下面的一些服务器均会被默认设置为$HOME_NETAIM_SERVERS设置为any

    Run

    Suricata的运行方式比较简单,只需要选择需要监听的网络接口,使用类似于下面的命令即可。

    sudo suricata -c /etc/suricata/suricata.yaml -i wlan0
    

    运行后产生的日志文件在/var/log/suricata目录下,我们可以使用类似于tail -f http.log stats.log的命令来监视程序的执行结果。

    Rules

    Suricata中最重要的就是关于规则的指定,使用特定的规则集就可以对特定的流量进行分析和处理,如果使用了IPS模式那么还可以直接处理报文内容。一般来说,我们会可以从互联网上下载最新的规则集,一般来自于Emerging Threats(Pro)和Sourcefire的VRT。手动管理的方式比较麻烦,我们可以使用到一款工具Oinkmasteroinkmaster可以自动化下载、管理rules。一般来说,规则由三个部分构成:Action、Header和Rule options。 例如下面这条

    alert tcp $EXTERNAL_NET any -> $HOME_NET 8888 (msg: "meow"; content: "meow"; )
    
    • alert表示动作,表示匹配后将发出警报。
    • tcp表示是TCP报文,还可以是ipudpicmp等,还包括一些常见的应用层协议。
    • $EXTERNAL表示使用前面定义的外部地址,可以使用!1.1.1.1![1.1.1.1, 1.1.1.2][10.0.0.0/24, !10.0.0.5]等形式。
    • any表示端口,有[79,80:82,83]这样的形式。
    • ->表示方向,可以是-><>
    • (msg: "meow"; content: "meow"; )表示规则选项,中间使用分号断开,包括meta-information、headers、payloads和flows等选项。具体内容将在后面说明。

    Meta-settings

    Meta-settings不会影响检测过程,只是用来完成记录等附属功能。

    msg: "some description"; 将显示在日志中
    sid: 123; 每条规则的编号
    rev: 123; 规则的版本号
    gid: 1; 组编号
    classtype: trojan-activity; 规则的分类
    reference: bugtraq, 123; http://www.securityfocus.com/bid; 规则的参考位置
    priority:1; 规则优先级
    metadata: ...;
    target: [src_ip|dest_ip];
    

    Header Keywords

    ttl: 10;
    ipopts: lsrr; IP选项
    sameip; 源IP和目的IP相同
    ip_proto: TCP;
    id: 1;
    geoip: src, RU;
    fragbits:[*+!]<[MDR]>;
    fragoffset:[!|<|>]<number>;
    seq:0;
    ack:1;
    window:[!]<number>;
    itype:min<>max;
    itype:[<|>]<number>;
    icode:min<>max;
    icode:[<|>]<number>;
    icmp_id:<number>;
    icmp_seq:<number>;
    

    Payload Keywords

    content:"a|0D|bc";
    content:"|61 0D 62 63|";
    content:"a|0D|b|63|";
    nocase;
    depth:12;
    offset:3;
    

    Flowbits

    通过在Suricata中保存标志位来判断若干个流量的关联性

    flowbits: set, name                设置name指定的条件
    flowbits: isset, name              检查是否有name指定的条件设置
    flowbits: toggle, name             切换name指定的条件设置情况
    flowbits: unset, name              取消设置name指定的条件
    flowbits: isnotset, name           检查是否没有name指定的条件设置
    flowbits: noalert                  不产生alert
    

    Flow

    匹配流的方向,是否建立连接等

    flow:to_client, established
    flow:to_server, established, only_stream
    flow:to_server, not_established, no_frag
    

    原理

    Suricata有几个关键组件构成:线程、线程模块和队列。Suricata以多线程的方式运行,而线程模块即对应其包获取、解码、检测和输出模块。一个包在Suricata会以类似流水线的方式一级一级地传递给下一个线程模块处理,而在这里的“传送带”就是队列。一个线程可以包含多个线程模块,这就是Runmode。 使用suricata --list-runmodes可以看到Suricata目前可以使用的runmodes。

    ------------------------------------- Runmodes ------------------------------------------
    | RunMode Type      | Custom Mode       | Description 
    |----------------------------------------------------------------------------------------
    | PCAP_DEV          | single            | Single threaded pcap live mode 
    |                   ---------------------------------------------------------------------
    |                   | autofp            | Multi threaded pcap live mode.  Packets from each flow are assigned to a single detect thread, unlike "pcap_live_auto" where packe
    ts from the same flow can be processed by any detect thread 
    |                   ---------------------------------------------------------------------
    |                   | workers           | Workers pcap live mode, each thread does all tasks from acquisition to logging 
    |----------------------------------------------------------------------------------------
    | PCAP_FILE         | single            | Single threaded pcap file mode 
    |                   ---------------------------------------------------------------------
    |                   | autofp            | Multi threaded pcap file mode.  Packets from each flow are assigned to a single detect thread, unlike "pcap-file-auto" where packe
    ts from the same flow can be processed by any detect thread 
    |----------------------------------------------------------------------------------------
    | PFRING(DISABLED)  | autofp            | Multi threaded pfring mode.  Packets from each flow are assigned to a single detect thread, unlike "pfring_auto" where packets fro
    m the same flow can be processed by any detect thread 
    |                   ---------------------------------------------------------------------
    |                   | single            | Single threaded pfring mode 
    |                   ---------------------------------------------------------------------
    |                   | workers           | Workers pfring mode, each thread does all tasks from acquisition to logging 
    |----------------------------------------------------------------------------------------
    | NFQ               | autofp            | Multi threaded NFQ IPS mode with respect to flow 
    |                   ---------------------------------------------------------------------
    |                   | workers           | Multi queue NFQ IPS mode with one thread per queue 
    |----------------------------------------------------------------------------------------
    | NFLOG             | autofp            | Multi threaded nflog mode   
    |                   ---------------------------------------------------------------------
    |                   | single            | Single threaded nflog mode  
    |                   ---------------------------------------------------------------------
    |                   | workers           | Workers nflog mode          
    |----------------------------------------------------------------------------------------
    | IPFW              | autofp            | Multi threaded IPFW IPS mode with respect to flow 
    |                   ---------------------------------------------------------------------
    |                   | workers           | Multi queue IPFW IPS mode with one thread per queue 
    |----------------------------------------------------------------------------------------
    | ERF_FILE          | single            | Single threaded ERF file mode 
    |                   ---------------------------------------------------------------------
    |                   | autofp            | Multi threaded ERF file mode.  Packets from each flow are assigned to a single detect thread 
    |----------------------------------------------------------------------------------------
    | ERF_DAG           | autofp            | Multi threaded DAG mode.  Packets from each flow are assigned to a single detect thread, unlike "dag_auto" where packets from the 
    same flow can be processed by any detect thread 
    |                   ---------------------------------------------------------------------
    |                   | single            | Singled threaded DAG mode   
    |                   ---------------------------------------------------------------------
    |                   | workers           | Workers DAG mode, each thread does all  tasks from acquisition to logging 
    |----------------------------------------------------------------------------------------
    | AF_PACKET_DEV     | single            | Single threaded af-packet mode 
    |                   ---------------------------------------------------------------------
    |                   | workers           | Workers af-packet mode, each thread does all tasks from acquisition to logging 
    |                   ---------------------------------------------------------------------
    |                   | autofp            | Multi socket AF_PACKET mode.  Packets from each flow are assigned to a single detect thread. 
    |----------------------------------------------------------------------------------------
    | NETMAP(DISABLED)  | single            | Single threaded netmap mode 
    |                   ---------------------------------------------------------------------
    |                   | workers           | Workers netmap mode, each thread does all tasks from acquisition to logging 
    |                   ---------------------------------------------------------------------
    |                   | autofp            | Multi threaded netmap mode.  Packets from each flow are assigned to a single detect thread. 
    |----------------------------------------------------------------------------------------
    | UNIX_SOCKET       | single            | Unix socket mode            
    |                   ---------------------------------------------------------------------
    |                   | autofp            | Unix socket mode            
    |----------------------------------------------------------------------------------------
    

    可以看到,在Suricata中包含三种Custom Mode,single/workers/autofp,根据右边的介绍我们能够知道当前模式的运行特点。在workers模式下,每一个线程上包含一个完整的包处理模块,也就是说将获取到的报文将分发到包处理线程中,而Suricata将会将属于同一个flow的流量放在一个线程中避免出现问题。

    其他支持软件

    Oinkmaster

    oinkmaster.pl -C /etc/oinkmaster.conf -o /etc/suricata/rules -i
    

    Suricata配置文件suricata.yaml中的outputs2 > unified2-alert可以设定在产生alert时dump出可疑数据包的信息,这个格式的好处是:

    • 方便归档管理
    • 生成速度快。

    Barnyard2

    Barnyard2就是个类似Syslog的东西,从Snort/Suricata处取得unified2格式的输入,产生其他格式的输出,比如给Prelude Hybrid IDS system、Syslog、MySQL。