[OpenWrt-Devel] OpenWRT: how to include byte-compile .pyc files into sysupgrade image?

[Xuebing Wang]

Hi Alex,

Thank you very much for your help. Apologies for the late response. I 
have been very busy recently. I do hope that I can find some time to 
backport these changes to CC 15.05.

xuebing

On 2017年03月10日 03:46, Alexandru Ardelean wrote:
> Hey Xuebing,
>
> Apologies for the late response.
> Short version: please checkout this PR
> https://github.com/openwrt/packages/pull/4130
> I am hoping you can backport these changes to OpenWrt CC 15.05
>
> I've managed to package Python & Python3 with .pyc bytecodes.
> And it seems to work fine. I tested with mpc85xx & x86_64, but please
> come back with a reply if it does not work for you and I can
> investigate.
> Maybe it's different for your architecture.
>
> That PR may be a bit more than you need, since it splits Python
> packages into Python Source packages and Python packages [ which now
> ship with .pyc files ].
> Funny story: I was approached about another user about this, but he
> was happy with just using  `python -m compileall`
>
> I tested both Python & Python3 versions.
> All looked good from what I could tell.
>
> Also, please notice anything in that PR that you could use to save
> some space on your flash device.
> I noticed that packages were shipping with test files ; I removed them.
>
> Hope this helps.
>
> Thanks
> Alex
>
> On Sat, Feb 18, 2017 at 1:26 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
>> Hi Alex,
>>
>> Thank you so much for your help.
>>
>> xuebing wang
>>
>>
>>
>> On 2017年02月17日 23:37, Alexandru Ardelean wrote:
>>> @Xuebing: so, there doesn't seem to be a very easy way to do this at build
>>> time.
>>> I mean, at first, it can seem easy, because all that is needed, is to
>>> add script that compiles here:
>>>
>>> https://github.com/openwrt/packages/blob/for-15.05/lang/python/files/python-package.mk#L94
>>>
>>> That script needs to recursively compile all .py files and
>>> [optionally]  remove them.
>>> So, feel free to try/hack something for your case.
>>>
>>> Maybe a shell script that uses this:
>>> http://effbot.org/zone/python-compile.htm
>>>
>>>> compile.py
>>> import os
>>> import sys
>>> import py_compile
>>> for file in sys.argv:
>>>       py_compile.compile(file)
>>>       os.remove(file)
>>>
>>> find . -name "\.py" | xargs compile.py
>>>
>>>
>>> For the general case [in trunk], I think I have to start considering
>>> splitting the Python source files and Python byte-compiled files into
>>> 2 package types.
>>> There was a older discussion about this ; the idea is to package the
>>> byte coded files, and offer the sources and python source packages.
>>> Example: Have `python` package and `python-sources`  package.
>>> `python` == byte-compiled .pyc files, `python-sources` .py files
>>>
>>> Will try to make an effort in the next couple of days to implement the
>>> general solution for this.
>>> But, it's Friday now here, and I usually don't do much in the weekend.
>>> So, hopefully next week I could allocate some time for this.
>>>
>>> Thanks
>>> Alex
>>>
>>> On Fri, Feb 17, 2017 at 9:42 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
>>>> Hi Alex,
>>>>
>>>> Thanks a lot for your help.  I am using Python 2.7.12, branch
>>>> chaos_calmer
>>>> from (git://github.com/openwrt/openwrt.git), feeds branch is for-15.05.
>>>>
>>>> Also, I am using Atheros platform based on MIPS architecture.
>>>>
>>>> Thanks again.
>>>>
>>>> xuebing wang
>>>>
>>>>
>>>> On 2017年02月17日 15:31, Alexandru Ardelean wrote:
>>>>> Oh.
>>>>>
>>>>> This is an old topic.
>>>>> Let me think about it, for a bit.
>>>>>
>>>>> Quick q: which Python version are you using and from which repo/branch ?
>>>>> I usually test against trunk.
>>>>>
>>>>> Thanks
>>>>> Alex
>>>>>
>>>>> On Fri, Feb 17, 2017 at 8:40 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
>>>>>> Hi Community and Alexandru:
>>>>>>
>>>>>> I am using OpenWrt chaos_calmer for an Atheros based platform, we have
>>>>>> a
>>>>>> complicated Python application which takes time to run.
>>>>>> - On the target board, I can "python -m compileall ." for Python
>>>>>> standard
>>>>>> libraries in directories "/usr/lib/python2.7", and boot time can
>>>>>> improve
>>>>>> significantly.
>>>>>>
>>>>>> Would you please advise how can I change in files
>>>>>> feeds/packages/lang/python
>>>>>> to byte-compile Python libraries and include .pyc files in the
>>>>>> sysupgrade
>>>>>> image?
>>>>>>
>>>>>> I am using below packages:
>>>>>> CONFIG_PACKAGE_python-base=y
>>>>>> CONFIG_PACKAGE_python-light=y
>>>>>> CONFIG_PACKAGE_python-codecs=y
>>>>>> CONFIG_PACKAGE_python-openssl=y
>>>>>> CONFIG_PACKAGE_python-sqlite3=y
>>>>>> CONFIG_PACKAGE_python-logging=y
>>>>>>
>>>>>> Thanks.
>>>>>> xuebing wang
>>>>>>
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel