patched
¶The most straightforward way to use the package is namely for patching. patched
allows you to replace one callable with another.
Let's see the
Suppose we have a Calculator
class:
class Calculator:
def eval(self, string):
return eval(string)
calc = Calculator()
calc.eval('3+4')
7
PatchSuite
is a suite of patches. The suite below consists of 1 patch:
import random
from patched import PatchSuite, patch
class BrokenCalc(PatchSuite):
@patch(parent=Calculator)
def eval(self, string):
value = eval(string)
random_inacuracy = 0.01 * value * random.random()
return value + 0.01 + random_inacuracy
with BrokenCalc():
print( calc.eval('3+4'))
7.010385867746241
parent
is the object being patched (a module, for example). The above patch suite definition is equivalent to this:
class BrokenCalc(PatchSuite):
class Meta:
parent = Calculator
@patch(wrapper_type=wrappers.Replacement)
def eval(self, string):
# ...
(Keyword) arguments declared in Meta
class are the default ones for each patch.
Replacement
wrapper type is the default one, so passing it has no effect. It means that the decorated function replaces the original function. The other available types are
Hook
: the decorated function is executed after the original one, the return value of the latter being known
Insertion
: inserted attribute is missing from the original parent object
You know the Counter
class from the collections
module:
import collections
collections.Counter('animal')
Counter({'a': 2, 'm': 1, 'i': 1, 'n': 1, 'l': 1})
Let's make it count only to 0:
from patched import wrappers
@patch(parent=collections, attribute='Counter')
class MyCounter(collections.Counter):
def __init__(self, *args, **kw):
c = collections.Counter(*args, **kw)
zeros_dict = dict.fromkeys(c.keys(), 0)
super().__init__(zeros_dict)
with PatchSuite([MyCounter.make_patch()]):
print( collections.Counter('animal'))
MyCounter({'m': 0, 'i': 0, 'n': 0, 'a': 0, 'l': 0})
The code seems to be totally different from the previous example. First: the callables being swapped are not functions, but classes (we replace Counter
with MyCounter
). It's not the common and recommended way, but for the purposes of this introductory section..
Second: patches are not collected from the testsuite declaration but passed to it explicitly. And somehow MyCounter
has make_patch
attribute..
Well, patch
is a class, and it's instances are meant to show "I want to make a patch out of this". It defines __call__()
method that accepts callable as a parameter which means patch
instance can be used as a decorator. Only patch
replaces the callable it decorates, so
type(MyCounter) == patch
True
MyCounter
{'wrapper_func': __main__.MyCounter, 'wrapper_type': patched.patching.wrappers.Replacement, 'attribute': 'Counter', 'parent': <module 'collections' from '/home/vitalii/.virtualenvs/gru3/lib/python3.4/collections/__init__.py'>}
Yeah, patch
inherits dict
.
As I said, patch
instance only marks an attribute as a future patch and can provide parameters to it. The real patch is constucted here:
MyCounter.make_patch()
Patch collections.Counter
type(_)
patched.patching.base.Patch
Note: inside the MyCounter
callable we have used collections.Counter(...)
and it really points to original callable there. That's because the patch is undone for the time of execution of our replacement callable. Respective lines from the source code:
load -r 31-35 ../patched/patching/wrappers.py
patch.off()
try:
return self.run(*args, **kwargs)
finally:
patch.on()
Previous example didn't actually stop Counter
from counting: it patched only it's __init__
method, but you can do all sorts of operations with counters.
Let's make a more
class StopCounting(PatchSuite):
class Meta:
parent = collections.Counter
@patch(wrapper_type=wrappers.Hook)
def _subtract_self(self, *args, _subtract=collections.Counter.subtract, **kw):
_subtract(self, **self)
@patch()
def _return_self(self, *args, **kw):
return self
__init__ = update = subtract = _subtract_self
__add__ = __sub__ = __or__ = __and__ = _return_self
del _subtract_self, _return_self
with StopCounting():
c = collections.Counter('animal')
c
union gives Counter({'a': 2, 'm': 1, 'i': 1, 'n': 1, 'l': 1})
c2 = collections.Counter('elephant')
c2
Counter({'e': 2, 'l': 1, 'h': 1, 't': 1, 'p': 1, 'a': 1, 'n': 1})
with StopCounting():
print('union gives %s' % (c | c2))
union gives Counter({'i': 0, 'm': 0, 'l': 0, 'a': 0, 'n': 0})
As you see, we are back with the initial style of declaring patches. All methods that update our instance subtract itself in the end, and all methods that return new instance return self. Patches _subtract_self
and _return_self
are deleted from class namespace and won't be collected.
Note: Also inside the replacement function we can get access to a lot of attributes from the respective event. For that we do write
@patch(pass_event=True, **other_kw)
def replacement_func(*args, event, **kw):
...
but that won't be covered now. Here is how it may look in practice:
class StopCounting(PatchSuite):
class Meta:
parent = collections.Counter
@patch(wrapper_type=wrappers.Hook)
def _subtract_self(self, *args, _subtract=collections.Counter.subtract, **kw):
_subtract(self, **self)
@patch(pass_event=True)
def _return_counter(self, *args, event, _subtract=collections.Counter.subtract, **kw):
ret = event.wrapped_func(self, *args, **kw)
_subtract(ret, **ret)
return ret
__init__ = update = subtract = _subtract_self
__add__ = __sub__ = __or__ = __and__ = _return_counter
del _subtract_self, _return_counter
c2 = collections.Counter('elephant')
c = collections.Counter('animal')
with StopCounting():
print('union gives %s' % (c | c2))
union gives Counter({'m': 0, 'l': 0, 'n': 0, 'i': 0, 'h': 0, 'e': 0, 't': 0, 'a': 0, 'p': 0})
These are actually not global, but threadlocal objects: they are global for this thread. To get the current instance you use .instance()
, to construct new instance you instantiate the object:
from patched.core.threadlocal import ThreadLocalMixin
class MyGlobal(ThreadLocalMixin):
global_name = "yet_another_global"
instance = MyGlobal.instance()
instance.var = 5
print( MyGlobal.instance().var)
MyGlobal.instance() == MyGlobal()
5
False
So, we have 3 global objects (patched.core.objects
):
Config
Yes, it's also a threadlocal object, not an absolute global. It can contain settings that are better to be reset: for example, events-breakpoints to stop at. In general, the Config
object doesn't play any significant role yet.
Storage
It's a hierarchical tree that stores data (in memory, just keeps links to objects, preventing them from deletion). It's straightforward to use: you put an object in it
from patched import get_storage
get_storage()['some.parameter'] = {'default': True}
and later fetch it:
get_storage()['some']
{'parameter': {'default': True}}
The most interesting one is
It stores LoggableEvent
s. For now the common case of it is the patched function that has been executed, but it can be absolutely different. This event is formatted to represent a corresponding line in the log, but it can also be accessed as a Python object, and introspected. But enough words, let's see some examples.
Firstly instantiate fresh new logger (reset all the previous records):
from patched.core.objects import Logger
new = Logger()
For the next set of examples we will use a web application, the official tutorial of Django REST framework
And one more thing: I have crafted an IPython extension to view the log, that is so short that I will paste it here:
load -r 5-18 ../patched/tools/ipython.py
@magics_class
class BlackMagics(Magics):
@line_magic
def events(self, line):
logger = Logger.instance()
if not line:
return logger
event = logger[int(line)]
return event
def load_ipython_extension(ip):
ip.register_magics(BlackMagics)
You can load it with the following command, or you can add it to IPython config to be loaded automatically.
load_ext patched.tools.ipython
Ok, let's start: you need django
and djangorestframework
to be installed
import os
os.chdir('../examples/rest-tutorial')
os.environ['DJANGO_SETTINGS_MODULE'] = 'tutorial.settings'
import django # if it's django >= 1.7, you need the next 2 lines
django.setup()
If you didn't know, django
has a test client that allows you to request your urls, and rest_framework
extends it:
from rest_framework.test import APIClient
client = APIClient()
client.login(username='vitalii', password='123')
True
The web application allows you to query existing code snippets and to create new ones. Following POST request creates a new snippet:
resp = client.post('/snippets/', {'title': 'my title', 'code': 'True = False'})
resp.data
{'url': 'http://testserver/snippets/5/', 'highlight': 'http://testserver/snippets/5/highlight/', 'owner': 'vitalii', 'title': 'my title', 'code': 'True = False', 'linenos': False, 'language': 'python', 'style': 'friendly'}
Let's see what's is it doing under the hood. let's add some logging. I've read from the documentation that it does the (de)serialization and that serializers have from_native
method:
from rest_framework import serializers as rest_serializers, fields as rest_fields
class SerializerPatch(PatchSuite):
from_native = patch(parent=rest_serializers.Serializer)
with SerializerPatch():
resp = client.post('/snippets/', {'title': 'my title', 'code': 'True = False'})
print(resp.data)
{'url': 'http://testserver/snippets/15/', 'highlight': 'http://testserver/snippets/15/highlight/', 'owner': 'vitalii', 'title': 'my title', 'code': 'True = False', 'linenos': False, 'language': 'python', 'style': 'friendly'}
events
Logged events: 0| from_native returned <Snippet: Snippet object>
events 0
from_native(self = <snippets.serializers.SnippetSerializer object at 0x7f53db3aced0>, data = <QueryDict: {'title': ['my title'], 'code': ['True = False']}>, files = <MultiValueDict: {}>, rv = Snippet object)
_.rv.code
'True = False'
As you see, it's not just a text line in the log. But let's get some more records, I've heard field values are obtained with field_from_native
method:
class WritableFieldPatch(PatchSuite):
field_from_native = patch(parent=rest_fields.WritableField, log_prefix='-> ')
Logger()
with SerializerPatch() + WritableFieldPatch():
resp = client.post('/snippets/', {'title': 'my title', 'code': 'True = False'})
print(resp.data)
{'url': 'http://testserver/snippets/12/', 'highlight': 'http://testserver/snippets/12/highlight/', 'owner': 'vitalii', 'title': 'my title', 'code': 'True = False', 'linenos': False, 'language': 'python', 'style': 'friendly'}
events
Logged events: 0| -> field_from_native returned None 1| -> field_from_native returned None 2| -> field_from_native returned None 3| -> field_from_native returned None 4| -> field_from_native returned None 5| from_native returned <Snippet: Snippet object>
events 3
field_from_native(self = <rest_framework.fields.ChoiceField object at 0x7f53db361850>, data = <QueryDict: {'title': ['my title'], 'code': ['True = False']}>, files = <MultiValueDict: {}>, field_name = language, into = {'style': 'friendly', 'title': 'my title', 'linenos': False, 'language': 'python', 'code': 'True = False'}, rv = None)
Oh yeah, field_from_native
simply puts the result into the into
dict, and returns None.
Let's make logging messages more informative (the code below is not meant to be understood right away):
from patched.patching.events import HookFunctionExecuted
from patched.patching import wrappers
from IPython.lib.pretty import pretty
class FieldFromNativeEvent(HookFunctionExecuted):
def _log_pretty_(self, p, cycle):
if cycle:
p.text('HookFunction(..)')
return
with p.group(len(self.log_prefix), self.log_prefix):
p.text( pretty(self.field_value))
p.text(' was set into ')
p.breakable()
p.text(self.field_name)
class WritableFieldPatch(PatchSuite):
@patch(wrapper_type=wrappers.Hook, event_class=FieldFromNativeEvent,
parent=rest_fields.WritableField, pass_event=True, log_prefix='-> ',
)
def field_from_native(self, data, files, field_name, into,
return_value, event):
event.field_name = field_name
event.field_value = into.get(field_name)
from patched.core.objects import Logger
Logger()
with SerializerPatch() + WritableFieldPatch():
resp = client.post('/snippets/', {'title': 'my title', 'code': 'True = False'})
print(resp.data)
{'url': 'http://testserver/snippets/18/', 'highlight': 'http://testserver/snippets/18/highlight/', 'owner': 'vitalii', 'title': 'my title', 'code': 'True = False', 'linenos': False, 'language': 'python', 'style': 'friendly'}
events
Logged events: 0| -> 'my title' was set into title 1| -> 'True = False' was set into code 2| -> False was set into linenos 3| -> 'python' was set into language 4| -> 'friendly' was set into style 5| from_native returned <Snippet: Snippet object>
Now it's all better.)
I can only compare patches
to mock
because I don't know any others. For now it is enough to say that they are very different: roughly speaking, mock
allows you to patch anything with anything. The functionality provided py patched
probably occupies there ten lines of code (I suspect I've seen that function). So, these two packages don't really intersect.
What it does intersect with.. esspecially the logging part of it - is the debugging utilities provided by python itself. For "smart logging" it probably would be better to make use of those.
So you should probably expect the package to become smaller, containing only the patching part, and some other packages to appear.