OpenStack 安装 LBaaS v2
- OpenStack 版本: Mitaka
- OpenStack Server OS: CentOS 7
LBaaS v2 adds the concept of listeners to the LBaaS v1 load balancers.
LBaaS v2 allows you to configure multiple listener ports on a single load balancer IP address.
网络节点：
安装包：
yum install openstack-neutron-lbaas修改配置文件 /etc/neutron/lbaas_agent.ini:
interface_driver = openvswitch # 'openvswitch' for Open vSwitch or 'linuxbridge' for Linux bridge.修改配置文件 /etc/neutron/neutron_lbaas.conf 或者 /etc/neutron/neutron.conf 中的 [service_providers]:
service_provider = LOADBALANCERV2:Haproxy:neutron_lbaas.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default 启动LBaaS服务:
systemctl enable neutron-lbaasv2-agent.service
systemctl start neutron-lbaasv2-agent.service 控制节点：
安装包:
yum install openstack-neutron-lbaas修改配置文件 /etc/neutron/neutron.conf:
service_plugins = [existing service plugins],neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2修改配置文件 /etc/neutron/neutron_lbaas.conf 或者 /etc/neutron/neutron.conf 中的 [service_providers]:
service_provider = LOADBALANCERV2:Haproxy:neutron_lbaas.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default 更新 Neutron 数据库
neutron-db-manage --subproject neutron-lbaas upgrade head重启 Neutron-Server 服务
systemctl restart neutron-server添加 LBaas 到 Dashboard
安装 dashboard plugin
git clone https://git.openstack.org/openstack/neutron-lbaas-dashboard
cd neutron-lbaas-dashboard
git checkout stable/mitaka
python setup.py install
cp neutron-lbaas-dashboard/enabled_1481_project_ng_loadbalancersv2_panel.py /usr/share/openstack-dashboard/openstack_dashboard/local/enabled/修改配置文件 /etc/openstack-dashboard/local_settings:
OPENSTACK_NEUTRON_NETWORK = {
...
'enable_lb': True
...CLI
创建 Load balancer :
neutron lbaas-loadbalancer-create \
[--name NAME] \
[--vip-address VIP_ADDRESS] \
VIP_SUBNET创建 Listener ：
neutron lbaas-listener-create \
[--connection-limit CONNECTION_LIMIT] \
[--name NAME] \
[--loadbalancer LOADBALANCER] \
--protocol {TCP,HTTP,HTTPS,TERMINATED_HTTPS} \
--protocol-port PORT创建 Pool :
neutron lbaas-pool-create \
[--name NAME] \
[--listener LISTENER] \
--lb-algorithm {ROUND_ROBIN,LEAST_CONNECTIONS,SOURCE_IP} \
--protocol {HTTP,HTTPS,TCP}创建 Member :
neutron lbaas-member-create \
[--weight WEIGHT] \
[--name NAME] \
--subnet SUBNET \
--address ADDRESS \
--protocol-port PROTOCOL_PORT \
POOL创建 Health monitor ：
neutron lbaas-healthmonitor-create \
[--name NAME] \
[--expected-codes EXPECTED_CODES] \
[--http-method HTTP_METHOD] \
[--url-path URL_PATH] \
--delay DELAY \
--max-retries MAX_RETRIES \
--timeout TIMEOUT \
--type {PING,TCP,HTTP,HTTPS}
--pool POOLVIP 绑定 Floating IP :
neutron floatingip-associate FLOATINGIP_ID LOAD_BALANCER_PORT_ID验证
创建 Load balancer :
neutron lbaas-loadbalancer-create --name LB-test bd10c417-e726-4bc2-ab1c-d8d62035189b
Created a new loadbalancer:
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| admin_state_up | True |
| description | |
| id | a19057c8-4180-4b3d-ab6b-227a81ec9785 |
| listeners | |
| name | LB-test |
| operating_status | OFFLINE |
| pools | |
| provider | haproxy |
| provisioning_status | PENDING_CREATE |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
| vip_address | 192.168.215.72 |
| vip_port_id | eb4732f7-7cd0-4b47-ad4f-f68ff86b6a3e |
| vip_subnet_id | bd10c417-e726-4bc2-ab1c-d8d62035189b |
+---------------------+--------------------------------------+HTTP Listener
创建 Listener :
neutron lbaas-listener-create --name HTTP --loadbalancer LB-test --protocol HTTP --protocol-port 80
Created a new listener:
+---------------------------+------------------------------------------------+
| Field | Value |
+---------------------------+------------------------------------------------+
| admin_state_up | True |
| connection_limit | -1 |
| default_pool_id | |
| default_tls_container_ref | |
| description | |
| id | 63059c51-e179-483a-bff7-51d55a6444b6 |
| loadbalancers | {"id": "a19057c8-4180-4b3d-ab6b-227a81ec9785"} |
| name | HTTP |
| protocol | HTTP |
| protocol_port | 80 |
| sni_container_refs | |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
+---------------------------+------------------------------------------------+创建 Pool :
neutron lbaas-pool-create --name HTTP --listener HTTP --lb-algorithm ROUND_ROBIN --protocol HTTP
Created a new pool:
+---------------------+------------------------------------------------+
| Field | Value |
+---------------------+------------------------------------------------+
| admin_state_up | True |
| description | |
| healthmonitor_id | |
| id | eb9ba09b-0bdd-4f77-9b1e-fe10aee3cf8f |
| lb_algorithm | ROUND_ROBIN |
| listeners | {"id": "63059c51-e179-483a-bff7-51d55a6444b6"} |
| loadbalancers | {"id": "a19057c8-4180-4b3d-ab6b-227a81ec9785"} |
| members | |
| name | HTTP |
| protocol | HTTP |
| session_persistence | |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
+---------------------+------------------------------------------------+创建 Member :
neutron lbaas-member-create --subnet bd10c417-e726-4bc2-ab1c-d8d62035189b --address 192.168.215.68 --protocol-port 80 HTTP
Created a new member:
+----------------+--------------------------------------+
| Field | Value |
+----------------+--------------------------------------+
| address | 192.168.215.68 |
| admin_state_up | True |
| id | e1df4d46-2f9b-4486-8b4f-559601343a8c |
| name | |
| protocol_port | 80 |
| subnet_id | bd10c417-e726-4bc2-ab1c-d8d62035189b |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
| weight | 1 |
+----------------+--------------------------------------+
neutron lbaas-member-create --subnet bd10c417-e726-4bc2-ab1c-d8d62035189b --address 192.168.215.69 --protocol-port 80 HTTP
Created a new member:
+----------------+--------------------------------------+
| Field | Value |
+----------------+--------------------------------------+
| address | 192.168.215.69 |
| admin_state_up | True |
| id | 19cf65bc-a3d9-4d8c-83f6-0e570de655aa |
| name | |
| protocol_port | 80 |
| subnet_id | bd10c417-e726-4bc2-ab1c-d8d62035189b |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
| weight | 1 |
+----------------+--------------------------------------+创建 Health monitor :
neutron lbaas-healthmonitor-create --expected-codes 200 --http-method GET --url-path / --timeout 5 --max-retries 3 --delay 5 --type HTTP --pool HTTP
Created a new healthmonitor:
+----------------+------------------------------------------------+
| Field | Value |
+----------------+------------------------------------------------+
| admin_state_up | True |
| delay | 5 |
| expected_codes | 200 |
| http_method | GET |
| id | 6b3465e6-2cec-475a-8bc0-11e83980b790 |
| max_retries | 3 |
| name | |
| pools | {"id": "eb9ba09b-0bdd-4f77-9b1e-fe10aee3cf8f"} |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
| timeout | 5 |
| type | HTTP |
| url_path | / |
+----------------+------------------------------------------------+SSH Listener
创建 Listener :
neutron lbaas-listener-create --name TCP --loadbalancer LB-test --protocol TCP --protocol-port 22
Created a new listener:
+---------------------------+------------------------------------------------+
| Field | Value |
+---------------------------+------------------------------------------------+
| admin_state_up | True |
| connection_limit | -1 |
| default_pool_id | |
| default_tls_container_ref | |
| description | |
| id | e5b630c9-5d7b-4c98-8c72-55b56b07a91d |
| loadbalancers | {"id": "a19057c8-4180-4b3d-ab6b-227a81ec9785"} |
| name | TCP |
| protocol | TCP |
| protocol_port | 22 |
| sni_container_refs | |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
+---------------------------+------------------------------------------------+创建 Pool :
neutron lbaas-pool-create --name TCP --listener TCP --lb-algorithm ROUND_ROBIN --protocol TCP
Created a new pool:
+---------------------+------------------------------------------------+
| Field | Value |
+---------------------+------------------------------------------------+
| admin_state_up | True |
| description | |
| healthmonitor_id | |
| id | e08cbc20-0224-4ee6-b581-933e117c6dac |
| lb_algorithm | ROUND_ROBIN |
| listeners | {"id": "e5b630c9-5d7b-4c98-8c72-55b56b07a91d"} |
| loadbalancers | {"id": "a19057c8-4180-4b3d-ab6b-227a81ec9785"} |
| members | |
| name | TCP |
| protocol | TCP |
| session_persistence | |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
+---------------------+------------------------------------------------+创建 Member :
neutron lbaas-member-create --subnet bd10c417-e726-4bc2-ab1c-d8d62035189b --address 192.168.215.68 --protocol-port 22 TCP
Created a new member:
+----------------+--------------------------------------+
| Field | Value |
+----------------+--------------------------------------+
| address | 192.168.215.68 |
| admin_state_up | True |
| id | d5b07ce6-f542-4918-86c8-a708b5860672 |
| name | |
| protocol_port | 22 |
| subnet_id | bd10c417-e726-4bc2-ab1c-d8d62035189b |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
| weight | 1 |
+----------------+--------------------------------------+
neutron lbaas-member-create --subnet bd10c417-e726-4bc2-ab1c-d8d62035189b --address 192.168.215.69 --protocol-port 22 TCP
Created a new member:
+----------------+--------------------------------------+
| Field | Value |
+----------------+--------------------------------------+
| address | 192.168.215.69 |
| admin_state_up | True |
| id | d830b716-93e4-4c42-acda-aec67b95696e |
| name | |
| protocol_port | 22 |
| subnet_id | bd10c417-e726-4bc2-ab1c-d8d62035189b |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
| weight | 1 |
+----------------+--------------------------------------+创建 Health monitor :
neutron lbaas-healthmonitor-create --timeout 5 --max-retries 3 --delay 5 --type TCP --pool TCP
Created a new healthmonitor:
+----------------+------------------------------------------------+
| Field | Value |
+----------------+------------------------------------------------+
| admin_state_up | True |
| delay | 5 |
| id | 00b947ee-7771-458a-8e02-047a6b724150 |
| max_retries | 3 |
| name | |
| pools | {"id": "e08cbc20-0224-4ee6-b581-933e117c6dac"} |
| tenant_id | ea46829ab10148c7b5c551e29798fa93 |
| timeout | 5 |
| type | TCP |
+----------------+------------------------------------------------+
从验证结果可以看到
- SSH 到 VIP 的流量被平均分配到了2个 Member
- HTTP 到 VIP 的流量被平均分配到了2个 Member
查看LBaaS进程:
ps -ef | grep lbaas
nobody 2018 1 0 10:48 ? 00:00:00 haproxy -f /var/lib/neutron/lbaas/v2/a19057c8-4180-4b3d-ab6b-227a81ec9785/haproxy.conf -p /var/lib/neutron/lbaas/v2/a19057c8-4180-4b3d-ab6b-227a81ec9785/haproxy.pid -sf 1772
neutron 16450 1 0 09:08 ? 00:00:16 /usr/bin/python2 /usr/bin/neutron-lbaasv2-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /usr/share/neutron/neutron-lbaas-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/lbaas_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-lbaasv2-agent --log-file /var/log/neutron/lbaas-agent.log可以看到HAproxy配置文件在cat /var/lib/neutron/lbaas/v2/a19057c8-4180-4b3d-ab6b-227a81ec9785/haproxy.conf
cat /var/lib/neutron/lbaas/v2/a19057c8-4180-4b3d-ab6b-227a81ec9785/haproxy.conf
# Configuration for LB-test
global
daemon
user nobody
group haproxy
log /dev/log local0
log /dev/log local1 notice
stats socket /var/lib/neutron/lbaas/v2/a19057c8-4180-4b3d-ab6b-227a81ec9785/haproxy_stats.sock mode 0666 level user
defaults
log global
retries 3
option redispatch
timeout connect 5000
timeout client 50000
timeout server 50000
frontend 63059c51-e179-483a-bff7-51d55a6444b6
option tcplog
option forwardfor
bind 192.168.215.72:80
mode http
default_backend eb9ba09b-0bdd-4f77-9b1e-fe10aee3cf8f
frontend e5b630c9-5d7b-4c98-8c72-55b56b07a91d
option tcplog
bind 192.168.215.72:22
mode tcp
default_backend e08cbc20-0224-4ee6-b581-933e117c6dac
backend eb9ba09b-0bdd-4f77-9b1e-fe10aee3cf8f
mode http
balance roundrobin
timeout check 5
option httpchk GET /
http-check expect rstatus 200
server e1df4d46-2f9b-4486-8b4f-559601343a8c 192.168.215.68:80 weight 1 check inter 5s fall 3
server 19cf65bc-a3d9-4d8c-83f6-0e570de655aa 192.168.215.69:80 weight 1 check inter 5s fall 3
backend e08cbc20-0224-4ee6-b581-933e117c6dac
mode tcp
balance roundrobin
timeout check 5
server d830b716-93e4-4c42-acda-aec67b95696e 192.168.215.69:22 weight 1 check inter 5s fall 3
server d5b07ce6-f542-4918-86c8-a708b5860672 192.168.215.68:22 weight 1 check inter 5s fall 3- frontend 下的 bind 192.168.215.72 就是VIP
- backend 下的 server 就是Member对象
- backend 下的 balance roundrobin 就是Pool对象的负载均衡方法
- 有多个 listener 和 pool 就有 多个 frontend 和 backend
Member Status Issue
BUG: https://bugs.launchpad.net/neutron/+bug/1464229
Patch: https://review.openstack.org/#/c/215027/4/neutron_lbaas/drivers/common/agent_callbacks.py
diff --git a/neutron_lbaas/drivers/common/agent_callbacks.py b/neutron_lbaas/drivers/common/agent_callbacks.py
index b03954f..6ef6a6a 100644
--- a/neutron_lbaas/drivers/common/agent_callbacks.py
+++ b/neutron_lbaas/drivers/common/agent_callbacks.py
@@ -5,29 +5,32 @@
5 5 # a copy of the License at
6 6 #
7 7 # http://www.apache.org/licenses/LICENSE-2.0
8 8 #
9 9 # Unless required by applicable law or agreed to in writing, software
10 10 # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
11 11 # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
12 12 # License for the specific language governing permissions and limitations
13 13 # under the License.
14 14
15 +import six
16 +
15 17 from neutron.extensions import portbindings
16 18 from neutron.plugins.common import constants
17 19 from neutron_lib import exceptions as n_exc
18 20 from oslo_log import log as logging
19 21 import oslo_messaging as messaging
20 22
21 23 from neutron_lbaas._i18n import _, _LW
22 24 from neutron_lbaas.db.loadbalancer import loadbalancer_dbv2
23 25 from neutron_lbaas.db.loadbalancer import models as db_models
26 +from neutron_lbaas.services.loadbalancer import constants as lb_const
24 27 from neutron_lbaas.services.loadbalancer import data_models
25 28
26 29 LOG = logging.getLogger(__name__)
27 30
28 31
29 32 class LoadBalancerCallbacks(object):
30 33
31 34 # history
32 35 # 1.0 Initial version
33 36 target = messaging.Target(version='1.0')
@@ -200,10 +203,29 @@
200 203 except n_exc.PortNotFound:
201 204 LOG.debug('Unable to find port %s to unplug. This can occur when '
202 205 'the Vip has been deleted first.',
203 206 port_id)
204 207
205 208 def update_loadbalancer_stats(self, context,
206 209 loadbalancer_id=None,
207 210 stats=None):
208 211 self.plugin.db.update_loadbalancer_stats(context, loadbalancer_id,
209 212 stats)
213 + # update members status
214 + if stats.get('members'):
215 + self.update_members_operating_status(context, stats['members'])'
216 +
217 + def update_members_operating_status(self, context, members):
218 + """Update members operating status from HAProxy statistics.
219 +
220 + HAProxy statistics also contain the current status of the servers.
221 + Update member's operating status from this current status.
222 + We don't update the current status when the member is DISABLED.
223 + """
224 + for mem_id, health_check in six.iteritems(members):
225 + status = lb_const.ONLINE
226 + if health_check['status'] != constants.ACTIVE:
227 + status = lb_const.OFFLINE
228 + mem = self.plugin.db.get_pool_member(context, mem_id)
229 + if mem.operating_status != lb_const.DISABLED:
230 + self.plugin.db.update_status(context, db_models.MemberV2,
231 + mem_id, operating_status=status)
你好，请问个问题，对haproxy的conf 模板在哪可以定制？比如我想修改 timeout connect 5000
timeout client 50000
timeout server 50000
这3个超时的参数，一直尝试修改.j2文件，但是重启neutron-lbaas的服务，配置就会恢复如初，请指教！
/usr/lib/python2.7/site-packages/neutron_lbaas/services/loadbalancer/drivers/haproxy/cfg.py 中 _build_defaults