[ACCEPTED]-Issue with ManyToMany Relationships not updating immediately after save-django-signals

When you save a model via admin forms it's 22 not an atomic transaction. The main object 21 gets saved first (to make sure it has a 20 PK), then the M2M is cleared and the new values 19 set to whatever came out of the form. So 18 if you are in the save() of the main object 17 you are in a window of opportunity where 16 the M2M hasn't been updated yet. In fact, if 15 you try to do something to the M2M, the change 14 will get wiped out by the clear(). I ran 13 into this about a year ago.

The code has 12 changed somewhat from the pre-ORM refactor 11 days, but it boils down to code in django.db.models.fields.ManyRelatedObjectsDescriptor and 10 ReverseManyRelatedObjectsDescriptor. Look at their __set__() methods and you'll 9 see manager.clear(); manager.add(*value) That clear() complete cleans out any 8 M2M references for the current main object 7 in that table. The add() then sets the new 6 values.

So to answer your question: yes, this 5 is a transaction issue.

Is there a signal 4 thrown when the transaction ends? Nothing 3 official, but read on:

There was a related thread a few months ago and MonkeyPatching 2 was one method proposed. Grégoire posted a MonkeyPatch for this. I haven't 1 tried it, but it looks like it should work.

When you are trying to access the ManyToMany 11 fields in the post_save signal of the model, the 10 related objects have already been removed 9 and will not be added again until after 8 the signal is finished.

To access this data 7 you have to tie into the save_related method 6 in your ModelAdmin. Unfortunately you'll 5 also have to include the code in the post_save 4 signal for non-admin requests that require 3 customization.

see: https://docs.djangoproject.com/en/1.7/ref/contrib/admin/#django.contrib.admin.ModelAdmin.save_related

Example:

# admin.py
Class GroupAdmin(admin.ModelAdmin):
    ...
    def save_related(self, request, form, formsets, change):
        super(GroupAdmin, self).save_related(request, form, formsets, change)
        # do something with the manytomany data from the admin
        form.instance.users.add(some_user)

Then in your 2 signals you can make the same changes that 1 you want to execute on a save:

# signals.py
@receiver(post_save, sender=Group)
def group_post_save(sender, instance, created, **kwargs):
    # do somethign with the manytomany data from non-admin
    instance.users.add(some_user)
    # note that instance.users.all() will be empty from the admin: []

I have a general solution to this that seems 10 a bit cleaner than monkey-patching the core 9 or even using celery (although I'm sure 8 someone could find areas where it fails). Basically 7 I add a clean() method in the admin for 6 the form that has the m2m relationships, and 5 set the instance relations to the cleaned_data 4 version. This make the correct data available 3 to the instance's save method, even though 2 it's not "on the books" yet. Try it and 1 see how it goes:

def clean(self, *args, **kwargs):
    # ... actual cleaning here
    # then find the m2m fields and copy from cleaned_data to the instance
    for f in self.instance._meta.get_all_field_names():
        if f in self.cleaned_data:
            field = self.instance._meta.get_field_by_name(f)[0]
            if isinstance(field, ManyToManyField):
                setattr(self.instance,f,self.cleaned_data[f])

See http://gterzian.github.io/Django-Cookbook/signals/2013/09/07/manipulating-m2m-with-signals.html

problem: When you manipulate the m2m 16 of a model within a post or pre_save signal 15 receiver, your changes get wiped out in 14 the subsequent 'clearing' of the m2m by 13 Django.

solution: In you post or pre_save 12 signal handler, register another handler 11 to the m2m_changed signal on the m2m intermediary 10 model of the model whose m2m you want to 9 update.

Please note that this second handler 8 will receive several m2m_changed signals, and 7 it is key to test for the value of the 'action' arguments 6 passed along with them.

Within this second 5 handler, check for the 'post_clear' action. When 4 you receive a signal with the post_clear 3 action, the m2m has been cleared by Django 2 and you have a chance to successfully manipulate 1 it.

an example:

def save_handler(sender, instance, *args, **kwargs):
    m2m_changed.connect(m2m_handler, sender=sender.m2mfield.through, weak=False)


def m2m_handler(sender, instance, action, *args, **kwargs):
    if action =='post_clear':
        succesfully_manipulate_m2m(instance)


pre_save.connect(save_handler, sender=YouModel, weak=False)

see https://docs.djangoproject.com/en/1.5/ref/signals/#m2m-changed

You can find more informations in this thread 1 : Django manytomany signals?

One of the solutions to update m2m, along 19 with updating one of your models.

Django 1.11 and higher

First of 18 all, all requests via admin panel are atomic. You 17 can look at ModelAdmin:

@csrf_protect_m
def changeform_view(self, request, object_id=None, form_url='', extra_context=None):
    with transaction.atomic(using=router.db_for_write(self.model)):
        return self._changeform_view(request, object_id, form_url, extra_context)

@csrf_protect_m
def delete_view(self, request, object_id, extra_context=None):
    with transaction.atomic(using=router.db_for_write(self.model)):
        return self._delete_view(request, object_id, extra_context)

The behavior which 16 you can observe during updating, when changes 15 which you made with m2m records were not 14 saved, even after you made them in a save 13 method one of your models or in a signal, happens 12 only because m2m form rewrites all records 11 after the main object is updated.

This is 10 why, step by step:

1. The main object is updated.

2. Your 9 code(in a save method or in a signal) made 8 changes (you can look at them, just put 7 a breakpoint in ModelAdmin):

 def save_related(self, request, form, formsets, change):
     breakpoint()
     form.save_m2m()
     for formset in formsets:
         self.save_formset(request, form, formset, change=change)

3. form.save_m2m() takes all m2m values which were placed on a page(roughly speaking) and replace all m2m records via a related manager. That's why you can't see your changes at the end of a transaction.

There is a solution: make 6 your changes with m2m via transaction.on_commit. transaction.on_commit 5 will make your changes after form.save_m2m() when 4 the transaction is committed.

Unfortunately, the 3 downside of this solution - your changes 2 with m2m will be executed in a separate 1 transaction.