python-varnishapi
python-varnishapi copied to clipboard
xcir
================== python-varnishapi
Connect to libvarnish api by ctypes
:Author: Shohei Tanaka(@xcir) :Date: 2018-03-19 :Version: 60.24 :Support Varnish Version: 4.0.x, 4.1.x 5.0.x 5.1.x 5.2.x 6.0.x 6.1.x :Check Varnish Version: 6.0.x 6.1.x :Check Python Version: 2.7.x, 3.5.x :Manual section: 3
For Varnish3.0.x
See this link. https://github.com/xcir/python-varnishapi/tree/varnish30
For Golang
See this link. https://github.com/xcir/go-varnishapi
Installation
sudo python setup.py install
Versioning
[varnish-version].[library-version]
50.18 is v18 for Varnish5.0.x
DESCRIPTION
Connect to libvarnish api by ctypes
VSLUtil class
VSLUtil.tag2Var
Prototype ::
tag2Var(tag, data)
Parameter ::
STRING tag
STRING data
Return value ::
DICT Var
Description ::
Transcode spec and tagname to variable name and value
Example ::
util = VSLUtil()
# output is {'val': ' example.net', 'key': 'resp.http.Host', 'vkey': 'resp'}
print util.tag2Var('RespHeader','Host: example.net')
VSLUtil.tag2VarName
Prototype ::
tag2VarName(tag, data)
Parameter ::
STRING tag
STRING data
Return value ::
STRING Variable name
Description ::
Transcode spec and tagname to variable name
Example ::
util = VSLUtil()
# output is "resp.http.Host"
print util.tag2VarName('RespHeader','Host: example.net')
VarnishStat class
VarnishStat.init
Prototype ::
VarnishStat(opt = '', sopath = 'libvarnishapi.so.1')
Parameter ::
LIST arg [OPTION]
STRING libvarnishapi path [OPTION]
Return value ::
class object
Description ::
initialize
Example ::
vsc = VarnishStat()
#set arg
vsc = VarnishStat(['-n', 'v2'])
VarnishStat.Fini
Prototype ::
Fini()
Parameter ::
VOID
Return value ::
VOID
Description ::
finish
Example ::
vsc = VarnishStat()
...
vsc.Fini()
VarnishStat.getStats
Prototype ::
getStats()
Parameter ::
VOID
Return value ::
DICT stats
Description ::
Get statistics counter
Example ::
vsc = varnishapi.VarnishStat()
r= vsc.getStat();
for k,v in r.iteritems():
#output
# MAIN.fetch_zero 0 Fetch zero len body
# MAIN.vmods 1 Loaded VMODs
# MAIN.sess_dropped 0 Sessions dropped for thread
# LCK.ban.locks 1457831 Lock Operations
#...
print "%40s %20s %s" % (k,v['val'],v['desc'])
VarnishLog class
This class is legacy.(using VSM) I recommend using VarnishLogVUT.
VarnishLog.init
Prototype ::
VarnishLog(opt = '', sopath = 'libvarnishapi.so.1', dataDecode = 1)
Parameter ::
LIST arg [OPTION]
STRING libvarnishapi path [OPTION]
INT Using decode at the callback [OPTION]
Return value ::
class object
Description ::
initialize
Example ::
vsl = VarnishLog()
#set arg
vsl = VarnishLog(['-n', 'v2'])
VarnishLog.Fini
Prototype ::
Fini()
Parameter ::
VOID
Return value ::
VOID
Description ::
finish
Example ::
vsl = VarnishLog()
...
vsl.Fini()
VarnishLog.Dispatch
Prototype ::
Dispatch(cb=None, priv=None, maxread=1, vxidcb=None, groupcb=None)
Parameter ::
FUNC cb callback function per line
OBJECT priv
INT maxread Maximum number of reads, if have unread log in VSL.(0=infinity)
FUNC vxidcb callback function per vxid(call per line, if group option set to raw)
FUNC groupcb callback function per group(raw, vxid, request, session)
===================== ======== ======== =========== =========== callbacktype \ group raw vxid request session ===================== ======== ======== =========== =========== cb per line per line per line per line vxidcb per line per vxid per vxid per vxid groupcb per line per vxid per request per session ===================== ======== ======== =========== ===========
Return value ::
INT
Description ::
Dispatch callback function
Example ::
def cbline(vap, cbd, priv):
print cbd
def cbvxid(vap, priv):
print "VXID"
def cbgroup(vap, priv):
print "GROUP"
vsl = varnishapi.VarnishLog(['-g','request'])
arg = {
'cb' : cbline,
'vxidcb' : cbvxid,
'groupcb' : cbgroup,
'maxread' : 0,
}
while 1:
ret = vsl.Dispatch(**arg)
if 0 == ret:
time.sleep(0.5)
vsl.Fini()
#output
#
# {'level': 1L, 'data': u'req 32907 rxreq', 'length': 16L, 'transaction_type': 2, 'reason': 2, 'tag': 76L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# {'level': 1L, 'data': u'Start: 1509598791.285514 0.000000 0.000000', 'length': 43L, 'transaction_type': 2, 'reason': 2, 'tag': 80L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# {'level': 1L, 'data': u'Req: 1509598791.285514 0.000000 0.000000', 'length': 41L, 'transaction_type': 2, 'reason': 2, 'tag': 80L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# ...
# {'level': 1L, 'data': u'', 'length': 1L, 'transaction_type': 2, 'reason': 2, 'tag': 77L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# VXID
# {'level': 2L, 'data': u'bereq 32908 fetch', 'length': 18L, 'transaction_type': 3, 'reason': 6, 'tag': 76L, 'vxid': 32909, 'vxid_parent': 32908, 'type': 'b', 'isbin': 0L}
# ...
# {'level': 2L, 'data': u'165 0 165 160 298 458', 'length': 22L, 'transaction_type': 3, 'reason': 6, 'tag': 83L, 'vxid': 32909, 'vxid_parent': 32908, 'type': 'b', 'isbin': 0L}
# {'level': 2L, 'data': u'', 'length': 1L, 'transaction_type': 3, 'reason': 6, 'tag': 77L, 'vxid': 32909, 'vxid_parent': 32908, 'type': 'b', 'isbin': 0L}
# VXID
# GROUP
# {'level': 1L, 'data': u'req 65648 rxreq', 'length': 16L, 'transaction_type': 2, 'reason': 2, 'tag': 76L, 'vxid': 65649, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# {'level': 1L, 'data': u'Start: 1509598842.452101 0.000000 0.000000', 'length': 43L, 'transaction_type': 2, 'reason': 2, 'tag': 80L, 'vxid': 65649, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
VarnishLog.VSL_tags / VSL_tags_rev
Prototype ::
#This is list variable
VSL_tags[tag index]
#This is dictionary variable
VSL_tags_rev[tag name]
Return value ::
STRING tagname (VSL_tags)
INT tagindex (VSL_tags_rev)
Description ::
Transcode tag index to tag text, or reverse
Example ::
def cb(vap,cbd,priv):
#output
#...
#VCL_call
#VCL_return
#...
print vap.VSL_tags[cbd['tag']]
vsl = varnishapi.VarnishLog(['-c'])
while 1:
ret = vsl.Dispatch(cb)
if 0 == ret:
time.sleep(0.5)
vsl.Fini()
VarnishLog.VSL_tagflags
Prototype ::
#This is list variable
VSL_tagflags[tag index]
Return value ::
INT tagflags
Description ::
tag flags
VarnishLog.VSLQ_grouping
Prototype ::
#This is list variable
VSLQ_grouping[tag index]
Return value ::
STRING VSLQ_grouping_name
Description ::
VSL Query grouping name
VarnishLogVUT class
Support Varnish5.2.0~(LIBVARNISHAPI2.0~)
VarnishLogVUT.init
Prototype ::
VarnishLogVUT(opt = [], progname='VarnishVUTproc', sopath = 'libvarnishapi.so.1', dataDecode = 1)
Parameter ::
LIST arg [OPTION]
STRING progname
STRING libvarnishapi path [OPTION]
INT Using decode at the callback [OPTION]
Return value ::
class object
Description ::
initialize
Example ::
arg = {}
arg["opt"] = ['-g','session']
vsl = VarnishLogVUT(**arg)
VarnishLogVUT.Fini
Prototype ::
Fini()
Parameter ::
VOID
Return value ::
VOID
Description ::
VarnishLogVUT is using thread.
Must call this function, if finish program.
Example ::
vsl = VarnishLogVUT()
...
vsl.Fini()
VarnishLogVUT.Dispatch
Prototype ::
Dispatch(cb=None, priv=None, maxread=1, vxidcb=None, groupcb=None)
Parameter ::
FUNC cb callback function per line
OBJECT priv
INT maxread Maximum number of reads, if have unread log in VSL.(0=infinity)
FUNC vxidcb callback function per vxid(call per line, if group option set to raw)
FUNC groupcb callback function per group(raw, vxid, request, session)
===================== ======== ======== =========== =========== callbacktype \ group raw vxid request session ===================== ======== ======== =========== =========== cb per line per line per line per line vxidcb per line per vxid per vxid per vxid groupcb per line per vxid per request per session ===================== ======== ======== =========== ===========
Return value ::
INT
Description ::
Dispatch callback function
Example ::
import signal
import varnishapi
import time
def cbline(vap, cbd, priv):
print cbd
def cbvxid(vap, priv):
print "VXID"
def cbgroup(vap, priv):
print "GROUP"
arg = {
'opt': ['-g','request']
}
vsl = varnishapi.VarnishLogVUT(**arg)
arg = {
'cb' : cbline,
'vxidcb' : cbvxid,
'groupcb' : cbgroup,
'maxread' : 0,
}
try:
vsl.Dispatch(**arg)
signal.pause()
except KeyboardInterrupt:
vsl.Fini()
#output
#
# {'level': 1L, 'data': u'req 32907 rxreq', 'length': 16L, 'transaction_type': 2, 'reason': 2, 'tag': 76L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# {'level': 1L, 'data': u'Start: 1509598791.285514 0.000000 0.000000', 'length': 43L, 'transaction_type': 2, 'reason': 2, 'tag': 80L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# {'level': 1L, 'data': u'Req: 1509598791.285514 0.000000 0.000000', 'length': 41L, 'transaction_type': 2, 'reason': 2, 'tag': 80L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# ...
# {'level': 1L, 'data': u'', 'length': 1L, 'transaction_type': 2, 'reason': 2, 'tag': 77L, 'vxid': 32908, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# VXID
# {'level': 2L, 'data': u'bereq 32908 fetch', 'length': 18L, 'transaction_type': 3, 'reason': 6, 'tag': 76L, 'vxid': 32909, 'vxid_parent': 32908, 'type': 'b', 'isbin': 0L}
# ...
# {'level': 2L, 'data': u'165 0 165 160 298 458', 'length': 22L, 'transaction_type': 3, 'reason': 6, 'tag': 83L, 'vxid': 32909, 'vxid_parent': 32908, 'type': 'b', 'isbin': 0L}
# {'level': 2L, 'data': u'', 'length': 1L, 'transaction_type': 3, 'reason': 6, 'tag': 77L, 'vxid': 32909, 'vxid_parent': 32908, 'type': 'b', 'isbin': 0L}
# VXID
# GROUP
# {'level': 1L, 'data': u'req 65648 rxreq', 'length': 16L, 'transaction_type': 2, 'reason': 2, 'tag': 76L, 'vxid': 65649, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
# {'level': 1L, 'data': u'Start: 1509598842.452101 0.000000 0.000000', 'length': 43L, 'transaction_type': 2, 'reason': 2, 'tag': 80L, 'vxid': 65649, 'vxid_parent': 0, 'type': 'c', 'isbin': 0L}
VarnishLogVUT.VSL_tags / VSL_tags_rev
Prototype ::
#This is list variable
VSL_tags[tag index]
#This is dictionary variable
VSL_tags_rev[tag name]
Return value ::
STRING tagname (VSL_tags)
INT tagindex (VSL_tags_rev)
Description ::
Transcode tag index to tag text, or reverse
Example ::
import signal
import varnishapi
import time
def cbline(vap, cbd, priv):
#output
#...
#VCL_call
#VCL_return
#...
print vap.VSL_tags[cbd['tag']]
arg = {
'opt': ['-c']
}
vsl = varnishapi.VarnishLogVUT(**arg)
arg = {
'cb' : cbline,
}
try:
vsl.Dispatch(**arg)
signal.pause()
except KeyboardInterrupt:
vsl.Fini()
VarnishLogVUT.VSL_tagflags
Prototype ::
#This is list variable
VSL_tagflags[tag index]
Return value ::
INT tagflags
Description ::
tag flags
VarnishLogVUT.VSLQ_grouping
Prototype ::
#This is list variable
VSLQ_grouping[tag index]
Return value ::
STRING VSLQ_grouping_name
Description ::
VSL Query grouping name
HISTORY
Version 60.24: Support VUT.(p-r #75 thanks ehocdet). Fix callback logic(vxid/groupcb). VarnishLog is deprecated, replace to VarnishLogVUT.
Version 52.23: Enhance perfomance. add some feature in dispatch(). add transaction_type in callbackdata(cbd)
Version 52.22: Fix VSC/varnishstat bug.(fix declare, add fini(I mis-deleted...) p-r #71,72 thanks ehocdet). Fix key varnishstat's decode for python3.
Version 52.21: Initial support Varnish5.2.x
Version 50.20: Improoved C binding.(p-r #67 thanks ema)
Version 50.19: Fix -c -b option issue.(issue #65 thanks ema)
Version 50.18: Support Varnish5.0 tags.
Version 40.17: Add VSL_TAG, VSL_DATA. Rename class from LIBVARNISHAPI13 to LIBVARNISHAPI.(p-r #56,57,58 thanks ehocdet)
Version 40.16: Change the decode error handler from "strict" to "replace".(p-r #51 thanks szymi-)
Version 40.15: Fix Crash.
Version 40.14: Fix decode issue.(via vsltrans p-r #25. thanks szymi-) Add dataDecode option in VarnishLog.init.
Version 40.13: No source change.
Version 40.12: Support pip.(p-r #39 thanks ziollek)
Version 0.11-varnish40: Initial support for Python3. Feedback is welcome.
Version 0.10-varnish40: Fix some error log did not output. (p-r #33 thanks ema)
Version 0.9-varnish40: Change VarnishLog.(VSL_tags|VSL_tagflags|VSLQ_grouping) from object to list. Add VarnishLog.VSL_tags_rev.
Version 0.8-varnish40: Fix Crash if log abandoned.
Version 0.7-varnish40: Support Varnish4.1 tags
Version 0.6-varnish40: Fix -n/-N option doesn't work in VarnishStat(issue #15 thanks athoune)
Version 0.5-varnish40: Add VSLUtil.tag2Var VarnishStat.Fini(p-r #10 thanks bryyyon)
Version 0.4-varnish40: Support change to Varnish4
Version 0.3-varnish30: Support VSM_ReOpen
Version 0.2-varnish30: Support VSL_Arg
Version 0.1-varnish30: First version
COPYRIGHT
python-varnishapi
- Copyright (c) 2015 Shohei Tanaka(@xcir)
Varnish Cache
- Copyright (c) 2006-2015 Varnish Software AS
xcir