IssueTracker

For the statement an app that mostly works in background does not mean its unimportant for the user, actually if it's important to the user, it should try to stay in the foreground, i.e., using a foreground service. We're squeezing the capability of running arbitrary amount of time for background apps. Look at the app restrictions since Android O, i.e., background services will be stopped by the system after 60s. So if an app is really offering critical functions to the user, it should try to stay in the foreground.

For the phantom processes, it's really a grey area, where the system does allow this kind of processes, but it's not formally supported, and it's not tracked by the system previously - it's going to hurt the overall system performance, as in resource-limited environment, other processes could be starving because of it. Thus we can't let the number of such kind of processes grow arbitrarily. The 32 may or may not be a good number, at least it's in par with the max cached processes, it's hard to justify what number is the perfect one though. I'd also be un-favor with the idea of introducing the per-package limit - from the system health point of view, it doesn't make sense to me.

For the battery optimizations disabled, this is orthogonal to the problem here I think. The battery optimization is more about the controls about background activities like Jobs/Alarms which could impact the battery negatively, while the problem here is more about other system resources such as CPU time and RAM.

For the cases like McAfee Security app, as you also show, the order of the killing is basically based on the parent process's oom adj score - if you app is killed before McAfee, that means it has a better oom adj score.

it should try to stay in the foreground, i.e., using a foreground service.

By background, I meant not visible like a social media app, and minimized to notifications. The TermuxService already is a foreground service and still its processes are getting killed. It normally has an oom adj of 200, while McAfee app also is a foreground service, but is usually at 100 oom adj, likely because of additional connections, and hence it survives, while Termux does not. Normally the heuristic of giving priority to McAfee would be fine-ish, but the heuristic is still just a guess at what is more important. What the OS's guess is of what's more important is not always what's more important for the user. An OS can't decide such a case. I ask a user, what he would rather have killed?, the Termux app that has important on-going work for him, which has session data that will be lost if its killed and can't be reproduced or like a backup operation going on that will have to restarted from scratch, costing with more CPU + battery + network bandwidth usage later, or would he rather have the McAfee app killed, that can just be restarted later and continue its monitoring again, without permanent data loss, since scans are reproducible. Even if loss was permanent for the later case, it may not be important to the user. I'm pretty sure the answer will be Termux for lot of users, if not most. I don't think one would hear many complaints of "Oh no, my antivirus got restarted and its scan data got lost!". Wouldn't it be better that the user gets to decide what the OS obviously can't decide. Moreover, as an app developer, the only choice would be to create useless connections to try to lower the oom odj to compete with MacAfee app, which would then waste OS resources and even that would be playing russian roulette.

The 32 may or may not be a good number, at least it's in par with the max cached processes, it's hard to justify what number is the perfect one though. I'd also be un-favor with the idea of introducing the per-package limit - from the system health point of view, it doesn't make sense to me.

How about, since 32 may not always be a good number, or will be different for each device or even each user, add a configurable option in android settings, or perhaps developer settings, like it is for cached processes, with a disable completely option as well for power users. Although, this doesn't solve the excessive usage killing issue.

it's going to hurt the overall system performance, as in resource-limited environment

I have a LG G5, with Snapdragon 820 and 4GB RAM + 2GB swap in /data partition, so its an old device but it runs apps relatively fine even today, with shell processes running through out the day. Some shell processes do take extra CPU and RAM usage, like daily compression+encryption backups, but those last a few minutes, and I as a user, I am completely fine with getting slow downs for a few minutes and don't want OS to kill such processes. I also have longer backups to the cloud, that run for over an hour during the night, with even more CPU usage, and they run without causing any user experience loss since I am not even using the phone and its on charging so battery is not an issue either. With the new design such things are not possible reliably. One would not want to wake up and find out that the OS decided to randomly kill their backup process during the night to save resources. I also use Activity State Changes Tasker Project that monitors every activity transition I do (in user space, not OS), and which happen quite often. Transitions are being monitored by multiple logcat processes and each transition runs dumpsys command as well to check fullscreen mode. I have never noticed any noticeable lag because of all that, in fact I have to put delays of like 1-2s because the tasks are much faster than required. The Tasker app barely even shows in battery usages in android settings. Other users on r/tasker have hundreds of event based tasks running all the time without any noticeable performance loss or battery usage.

So the argument that phantom processes will harm user experience is not always true. The r/tasker and r/termux users have actively been using such processes without issue for years. Moreover, every user may not even want an even faster OS/device, but a reliable one that is actually useful for his daily work, instead of just being faster. Devices are fast enough today, with batteries large enough too.

I am also not sure how many android OS devs experiment with such stuff in their daily life, who can provide more usable stats on this, even I don't get much time now. Do you guys have any such stats collected from such devs or done as part of development itself, like how killing normal phantom processes (not rogue) of apps like Termux and Tasker actually noticeably improved user experience and system performance for other apps. Those would be interesting to look at.

Yes, I completely agree that phantom processes can decrease system performance in some cases, specially like the case above with McAfee, but the user should get to decide if he is fine with a certain app causing a performance loss, hence the request for an exemption. Moreover, the current design in fact trimmed rogue logcat processes from 100+ to 32, but it also killed the bash process that wasn't even using much CPU and was useful to the user and the design prevented termux from actually running anything, so from a user perspective, the collateral damage is too high.

The battery optimization is more about the controls about background activities like Jobs/Alarms which could impact the battery negatively, while the problem here is more about other system resources such as CPU time and RAM.

Okay, fair enough. But from a user point of view, more CPU usage would be proportional to battery usage and it would have been easier to use the same permission. But if you want to introduce another permission "Disable resource optimizations", that would be acceptable too. Moreover, the apps that already had battery optimization disabled would automatically also get to use CPU unconditionally and bypass phantom process restriction if same permission was used, so that is probably not something one may want and a separate permission is likely more ideal.

The permission check could be added to ActivityManagerService.checkExcessivePowerUsage() before monitoring CPU usage and before bookeeping a phantom process in PhantomProcessList.getOrCreatePhantomProcessIfNeededLocked(). It would be really great if such an exemption was added for specific apps. The power users would be really grateful that they get to live another day and enjoy their device like they want to.

In terms of the oom adj score McAfee vs. Termux, I guess McAfee has a binding from a higher priority processes, such as the system. As you know, the Android OS doesn't have the intelligence to determine which one is important to the user - it has no idea what this app is for, an user could say the Termux is more important to them, while the other user could say McAfee is more important. It's not scalable for the Android OS to ask user in terms of the importance for every single app. Remember, user perception/interaction is a resource too.

The number 32 is AOSP definition, OEMs could alter this number if they like, there is no any hard requirement on this. I'd say we could explore the option to add a configuration in the developer options for it. But again, this kind of thing is never formally supported as an Android feature, I'm not sure if this Settings makes sense in this context.

However I'd disagree to associate it with the battery optimization settings, as aforementioned, they're orthogonal problems. Actually for most of the users, they don't have an idea about what the turning off battery optimization would actually mean to them. Apps without battery restrictions, don't mean they could do whatever they want. All of the Android OS users, should be protected from buggy or malicious apps - we do have seen buggy apps with native executables eating up the system resources. I'd also disagree with the statement Devices are fast enough today, with batteries large enough too, it's never our goal of stopping making faster device and longer battery life.

has a binding from a higher priority processes

Yes, that's what I was referring to, check the ProcessRecord entries which were posted in linked issue.

McAfee ProcessRecord

$ adb shell "/system/bin/dumpsys activity processes | grep -E -A 50 'ProcessRecord\{.*com.wsandroid.suite'"
  *APP* UID 10164 ProcessRecord{24e34c1 3461:com.wsandroid.suite/u0a164}
    user #0 uid=10164 gids={3002, 3003, 3001, 50164, 20164, 9997}
    mRequiredAbi=arm64-v8a instructionSet=arm64
    class=com.wsandroid.suite.MMSApplication
    manageSpaceActivityName=com.wavesecure.activities.ManageSpaceActivity
    dir=/data/app/~~iWO3mRIXmB_uMRsr0gJvYw==/com.wsandroid.suite-IQCqZvqHzO_TB7Dqs4PPsw==/base.apk publicDir=/data/app/~~iWO3mRIXmB_uMRsr0gJvYw==/com.wsandroid.suite-IQCqZvqHzO_TB7Dqs4PPsw==/base.apk data=/data/user/0/com.wsandroid.suite
    packageList={com.wsandroid.suite}
    packageDependencies={com.google.android.webview}
    compat={440dpi}
    thread=android.app.IApplicationThread$Stub$Proxy@2d1b295
    pid=3461
    lastActivityTime=-1m3s99ms
    startSeq=41
    mountMode=DEFAULT
    lastPssTime=-1m14s752ms pssProcState=4 pssStatType=1 nextPssTime=+58m44s597ms
    lastPss=210MB lastSwapPss=41MB lastCachedPss=0.00 lastCachedSwapPss=0.00 lastRss=220MB
    trimMemoryLevel=0
    procStateMemTracker: best=1 (1=1 Infinityx)
    lastRequestedGc=-15h6m1s363ms lastLowMemory=-15h10m20s35ms reportLowMemory=false
    reportedInteraction=false fgInteractionTime=-58s379ms
    adjSeq=647074 lruSeq=194598
    oom adj: max=1001 curRaw=100 setRaw=100 cur=100 set=100
    mCurSchedGroup=2 setSchedGroup=2 systemNoUi=false
    curProcState=4 mRepProcState=4 setProcState=4 lastStateTime=-2d8h39m55s697ms
    curCapability=LCMN setCapability=LCMN
    allowStartFgsState=4
    hasShownUi=true pendingUiClean=true
    cached=false empty=true
    notCachedSinceIdle=true initialIdlePss=81684
    lastTopTime=-58s378ms
    lastInvisibleTime=2021-11-01 20:53:45.546 (-58s385ms)
    hasStartedServices=true
    mHasForegroundServices=true forcingToImportant=null
    Services:
      - ServiceRecord{15396e2 u0 com.wsandroid.suite/com.mcafee.monitor.MMSAccessibilityService}
      - ServiceRecord{45bf48f u0 com.wsandroid.suite/com.mcafee.messaging.google.GoogleFCMMessageReceiver}
      - ServiceRecord{a1e649e u0 com.wsandroid.suite/com.mcafee.sdk.wp.core.siteadvisor.service.SiteAdvisorService}
      - ServiceRecord{c0ba211 u0 com.wsandroid.suite/com.mcafee.broadcast.MMSSystemBroadcastListencerService}
    mConnections:
      - ConnectionRecord{1036c33 u0 CR com.wsandroid.suite/org.chromium.content.app.SandboxedProcessService0:0:@1ff27a2}
      - ConnectionRecord{109c957 u0 CR WACT CAPS com.google.android.gms/.chimera.PersistentDirectBootAwareApiService:@305f2d6}
      - ConnectionRecord{6cc1bee u0 CR WPRI com.wsandroid.suite/org.chromium.content.app.SandboxedProcessService0:0:@ba02569}
      - ConnectionRecord{b9567ee u0 CR IMP com.wsandroid.suite/com.mcafee.messaging.google.GoogleFCMMessageReceiver:@3cf2169}
    Published Providers:
      - androidx.lifecycle.ProcessLifecycleOwnerInitializer
        -> ContentProviderRecord{bf09faa u0 com.wsandroid.suite/androidx.lifecycle.ProcessLifecycleOwnerInitializer}
      - com.mcafee.core.provider.KeyValueDataProvider
        -> ContentProviderRecord{d95f79b u0 com.wsandroid.suite/com.mcafee.core.provider.KeyValueDataProvider}
      - com.mcafee.core.provider.JSONDataContentProvider
        -> ContentProviderRecord{72dd738 u0 com.wsandroid.suite/com.mcafee.core.provider.JSONDataContentProvider}
      - com.moengage.core.internal.storage.database.MoEProvider

Termux ProcessRecord

$ adb shell "/system/bin/dumpsys activity processes | grep -E -A 50 'ProcessRecord\{.*com.termux'"
  *APP* UID 10383 ProcessRecord{f26c83c 18144:com.termux/u0a383}
    user #0 uid=10383 gids={3003, 50383, 20383, 9997}
    mRequiredAbi=arm64-v8a instructionSet=arm64
    class=com.termux.app.TermuxApplication
    dir=/data/app/~~gQtDuMCK77ed6DPdCmeWAA==/com.termux-MfgXPeqANrvwr29oJKqfNA==/base.apk publicDir=/data/app/~~gQtDuMCK77ed6DPdCmeWAA==/com.termux-MfgXPeqANrvwr29oJKqfNA==/base.apk data=/data/user/0/com.termux
    packageList={com.termux}
    compat={440dpi}
    thread=android.app.IApplicationThread$Stub$Proxy@f4fbbd
    pid=18144
    lastActivityTime=-53s621ms
    startSeq=4028
    mountMode=DEFAULT
    lastPssTime=-59s212ms pssProcState=4 pssStatType=1 nextPssTime=+4m4s71ms
    lastPss=20MB lastSwapPss=379KB lastCachedPss=0.00 lastCachedSwapPss=0.00 lastRss=57MB
    trimMemoryLevel=0
    procStateMemTracker: best=1 (1=1 7.59375x)
    lastRequestedGc=-9m30s398ms lastLowMemory=-9m30s398ms reportLowMemory=false
    reportedInteraction=false fgInteractionTime=-50s903ms
    adjSeq=647062 lruSeq=194591
    oom adj: max=1001 curRaw=200 setRaw=200 cur=200 set=200
    mCurSchedGroup=2 setSchedGroup=2 systemNoUi=false
    curProcState=4 mRepProcState=4 setProcState=4 lastStateTime=-9m30s394ms
    curCapability=-CMN setCapability=-CMN
    allowStartFgsState=4
    hasShownUi=true pendingUiClean=true
    cached=false empty=true
    lastTopTime=-50s903ms
    lastInvisibleTime=2021-11-01 20:53:37.304 (-50s903ms)
    hasStartedServices=true
    mHasForegroundServices=true forcingToImportant=null
    Services:
      - ServiceRecord{ecab86c u0 com.termux/.app.TermuxService}
    mConnections:
      - ConnectionRecord{44ab2ca u0 com.termux/.app.TermuxService:@99e35}
    Published Providers:
      - com.termux.filepicker.TermuxDocumentsProvider
        -> ContentProviderRecord{e30f5b2 u0 com.termux/.filepicker.TermuxDocumentsProvider}
      - com.termux.app.TermuxOpenReceiver$ContentProvider
        -> ContentProviderRecord{f610903 u0 com.termux/.app.TermuxOpenReceiver$ContentProvider}
    Connected Providers:
      - 8f1e68c/com.android.providers.settings/.SettingsProvider->18144:com.termux/u0a383 s1/1 u0/0 +9m30s337ms
    lastCompactTime=0 lastCompactAction=0
    isFreezeExempt=false isPendingFreeze=false isFrozen=false
    Activities:
      - ActivityRecord{aa40b10 u0 com.termux/.app.TermuxActivity t270}
    Recent Tasks:
      - Task{eb62c28 #270 type=standard A=10383:com.termux U=0 visible=false mode=fullscreen translucent=true sz=1}
    BoundClientUids:[10383]
     Configuration={1.0 310mcc150mnc [en_US,es_US,nb_NO] ldltr sw392dp w392dp h767dp 440dpi nrml long port night finger -keyb/v/h -nav/h winConfig={ mBounds=Rect(0, 0 - 1080, 2220) mAppBounds=Rect(0, 0 - 1080, 2176) mMaxBounds=Rect(0, 0 - 1080, 2220) mWindowingMode=fullscreen mDisplayWindowingMode=fullscreen mActivityType=undefined mAlwaysOnTop=undefined mRotation=ROTATION_0} as.2 s.6012 fontWeightAdjustment=0}
     OverrideConfiguration={1.0 310mcc150mnc [en_US,es_US,nb_NO] ldltr sw392dp w392dp h767dp 440dpi nrml long port night finger -keyb/v/h -nav/h winConfig={ mBounds=Rect(0, 0 - 1080, 2220) mAppBounds=Rect(0, 0 - 1080, 2176) mMaxBounds=Rect(0, 0 - 1080, 2220) mWindowingMode=fullscreen mDisplayWindowingMode=fullscreen mActivityType=standard mAlwaysOnTop=undefined mRotation=ROTATION_0} as.2 s.1 fontWeightAdjustment=0}
     mLastReportedConfiguration={1.0 310mcc150mnc [en_US,es_US,nb_NO] ldltr sw392dp w392dp h767dp 440dpi nrml long port night finger -keyb/v/h -nav/h winConfig={ mBounds=Rect(0, 0 - 1080, 2220) mAppBounds=Rect(0, 0 - 1080, 2176) mMaxBounds=Rect(0, 0 - 1080, 2220) mWindowingMode=fullscreen mDisplayWindowingMode=fullscreen mActivityType=undefined mAlwaysOnTop=undefined mRotation=ROTATION_0} as.2 s.6012 fontWeightAdjustment=0}

an user could say the Termux is more important to them, while the other user could say McAfee is more important. I

Yes, exactly. That is exactly the situation a permission would help in. One would grant permission to Termux, the other to McAfee. User decides for android on who to prioritize. If both have permission, then oom adj can be used as fallback. Of course, that situation wouldn't occur in context of phantom processes, since phantom process records wouldn't be tracked if permission was granted.

It's not scalable for the Android OS to ask user in terms of the importance for every single app.

Android does not need to ask. Only a runtime setting needs to exist and apps that are phantom process oriented can ask the user to grant them. Battery optimization permission exists too after all and so does draw over apps and these are already being used by such apps without complaints from users, other than OEM killers of course.

I'd say we could explore the option to add a configuration in the developer options for it.

That would be great and at least would be something. However, a permission exemption would still be better a way in my opinion because if its based on permission, then only specific apps will get to run phantom processes that were explicitly granted to do so and possibly trusted to be reliable. The other apps would still be restricted to the 32 limit and issue of rogue/buggy processes wasting system would be handled as well, which will improve user experience and performance, which is apparently your desired goal. If a configurable limit is used, then it would apply to all apps, and the rogue apps would be allowed to waste resources and users increasing the limit will suffer. But if the permission is not going to be added, then a configuration settings would be greatly appreciated, since at least adb or root will not be required, which would be better for users and devs.

Note that com.google.android.gms overrides the device config received from google servers after BOOT_COMPLETED event, as mentioned in the Why device config may get reset? section, so that case would need to be handled too, so that user setting is not overridden. Currently, that is also the issue with setting max_phantom_processes to 2147483647, since config gets overridden at boot, and user needs to set max_phantom_processes after that happens on every boot or has to disable config sync, which is of course not advisable for normal users. Moreover, even max_cached_processes value set in developer options does not survive reboots, since mOverrideMaxCachedProcesses gets set to -1 at boot. So asking user to set a value on every boot from developer options is not good from a usability perspective.

But again, this kind of thing is never formally supported as an Android feature

Background processes and daemons are part of linux kernel, like the execvp. If android still doesn't support them, maybe it is time it did. CPU performance and memory has greatly increased over last 10 years that it may be time do so if explicitly allowed by the user. Nothing wrong with opening things up sometimes, instead of locking them up.

Apps without battery restrictions, don't mean they could do whatever they want.

Sure, but killing processes like in case of phantom processes when there is no significant CPU usage or memory pressure is not good either. Killing them under memory pressure based on their oom adj when the time comes is perfectly fine. But before that, if the user wants them to, let them keep running in background and use the CPU as needed.

it's never our goal of stopping making faster device and longer battery life.

Yeah definitely, don't stop improving. I completely agree. I meant in the context that things are fast enough that a user may be satisfied with current levels of performance. If something gets done currently in 100ms and after optimizations it gets done in 80ms, he may still be fine with not having that extra boost, if he has to give up customizability and freedom for it. Users are often also not upgrading their devices every year to newer models, since a lot of them are fine with the performance of their device.