[OpenWrt-Devel] OpenWRT: how to include byte-compile .pyc files into sysupgrade image?

[Alexandru Ardelean]

Well, you might be able to just copy the package files over and it may
just work.
Not sure.

On Mon, Mar 27, 2017 at 10:01 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
> Hi Alex,
>
> Thank you very much for your help. Apologies for the late response. I have
> been very busy recently. I do hope that I can find some time to backport
> these changes to CC 15.05.
>
> xuebing
>
>
> On 2017年03月10日 03:46, Alexandru Ardelean wrote:
>>
>> Hey Xuebing,
>>
>> Apologies for the late response.
>> Short version: please checkout this PR
>> https://github.com/openwrt/packages/pull/4130
>> I am hoping you can backport these changes to OpenWrt CC 15.05
>>
>> I've managed to package Python & Python3 with .pyc bytecodes.
>> And it seems to work fine. I tested with mpc85xx & x86_64, but please
>> come back with a reply if it does not work for you and I can
>> investigate.
>> Maybe it's different for your architecture.
>>
>> That PR may be a bit more than you need, since it splits Python
>> packages into Python Source packages and Python packages [ which now
>> ship with .pyc files ].
>> Funny story: I was approached about another user about this, but he
>> was happy with just using  `python -m compileall`
>>
>> I tested both Python & Python3 versions.
>> All looked good from what I could tell.
>>
>> Also, please notice anything in that PR that you could use to save
>> some space on your flash device.
>> I noticed that packages were shipping with test files ; I removed them.
>>
>> Hope this helps.
>>
>> Thanks
>> Alex
>>
>> On Sat, Feb 18, 2017 at 1:26 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
>>>
>>> Hi Alex,
>>>
>>> Thank you so much for your help.
>>>
>>> xuebing wang
>>>
>>>
>>>
>>> On 2017年02月17日 23:37, Alexandru Ardelean wrote:
>>>>
>>>> @Xuebing: so, there doesn't seem to be a very easy way to do this at
>>>> build
>>>> time.
>>>> I mean, at first, it can seem easy, because all that is needed, is to
>>>> add script that compiles here:
>>>>
>>>>
>>>> https://github.com/openwrt/packages/blob/for-15.05/lang/python/files/python-package.mk#L94
>>>>
>>>> That script needs to recursively compile all .py files and
>>>> [optionally]  remove them.
>>>> So, feel free to try/hack something for your case.
>>>>
>>>> Maybe a shell script that uses this:
>>>> http://effbot.org/zone/python-compile.htm
>>>>
>>>>> compile.py
>>>>
>>>> import os
>>>> import sys
>>>> import py_compile
>>>> for file in sys.argv:
>>>>       py_compile.compile(file)
>>>>       os.remove(file)
>>>>
>>>> find . -name "\.py" | xargs compile.py
>>>>
>>>>
>>>> For the general case [in trunk], I think I have to start considering
>>>> splitting the Python source files and Python byte-compiled files into
>>>> 2 package types.
>>>> There was a older discussion about this ; the idea is to package the
>>>> byte coded files, and offer the sources and python source packages.
>>>> Example: Have `python` package and `python-sources`  package.
>>>> `python` == byte-compiled .pyc files, `python-sources` .py files
>>>>
>>>> Will try to make an effort in the next couple of days to implement the
>>>> general solution for this.
>>>> But, it's Friday now here, and I usually don't do much in the weekend.
>>>> So, hopefully next week I could allocate some time for this.
>>>>
>>>> Thanks
>>>> Alex
>>>>
>>>> On Fri, Feb 17, 2017 at 9:42 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
>>>>>
>>>>> Hi Alex,
>>>>>
>>>>> Thanks a lot for your help.  I am using Python 2.7.12, branch
>>>>> chaos_calmer
>>>>> from (git://github.com/openwrt/openwrt.git), feeds branch is for-15.05.
>>>>>
>>>>> Also, I am using Atheros platform based on MIPS architecture.
>>>>>
>>>>> Thanks again.
>>>>>
>>>>> xuebing wang
>>>>>
>>>>>
>>>>> On 2017年02月17日 15:31, Alexandru Ardelean wrote:
>>>>>>
>>>>>> Oh.
>>>>>>
>>>>>> This is an old topic.
>>>>>> Let me think about it, for a bit.
>>>>>>
>>>>>> Quick q: which Python version are you using and from which repo/branch
>>>>>> ?
>>>>>> I usually test against trunk.
>>>>>>
>>>>>> Thanks
>>>>>> Alex
>>>>>>
>>>>>> On Fri, Feb 17, 2017 at 8:40 AM, Xuebing Wang <xbing6 at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Community and Alexandru:
>>>>>>>
>>>>>>> I am using OpenWrt chaos_calmer for an Atheros based platform, we
>>>>>>> have
>>>>>>> a
>>>>>>> complicated Python application which takes time to run.
>>>>>>> - On the target board, I can "python -m compileall ." for Python
>>>>>>> standard
>>>>>>> libraries in directories "/usr/lib/python2.7", and boot time can
>>>>>>> improve
>>>>>>> significantly.
>>>>>>>
>>>>>>> Would you please advise how can I change in files
>>>>>>> feeds/packages/lang/python
>>>>>>> to byte-compile Python libraries and include .pyc files in the
>>>>>>> sysupgrade
>>>>>>> image?
>>>>>>>
>>>>>>> I am using below packages:
>>>>>>> CONFIG_PACKAGE_python-base=y
>>>>>>> CONFIG_PACKAGE_python-light=y
>>>>>>> CONFIG_PACKAGE_python-codecs=y
>>>>>>> CONFIG_PACKAGE_python-openssl=y
>>>>>>> CONFIG_PACKAGE_python-sqlite3=y
>>>>>>> CONFIG_PACKAGE_python-logging=y
>>>>>>>
>>>>>>> Thanks.
>>>>>>> xuebing wang
>>>>>>>
>
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel