headers
Mark Day | 1 Mar 2006 18:48
Picon
Favicon

Add a file flag for "hidden" files?

Hello,

For an future version of Mac OS X, I'm trying to solve a problem  
where certain files and directories on Windows file systems don't  
normally appear in the Windows GUI, but currently do appear in the  
Mac OS X GUI (but they shouldn't).  I'd like to gauge interest in  
having the change ported back to FreeBSD (since many of our file  
systems are derived from FreeBSD).  The approach I'm planning to use  
is to add a new flag to the file flags (i.e. the st_flags field from  
stat(2); settable via chflags(2)).  I'd like to avoid a collision  
with any flags FreeBSD would add in the future, so either have  
FreeBSD adopt the flag, or start allocating Mac OS X-specific flags  
from the high order bits (since FreeBSD has been allocating them from  
the low order bits).  My current name for this new file flag is  
UF_HIDDEN.

Windows file systems (FAT, NTFS, SMB) have a "hidden" attribute bit  
on files and directories.  If a file or directory has this attribute  
set, Windows will not normally show it in the GUI unless you explicit  
ask to view hidden items.  Mac file systems (HFS, AFP) have a similar  
notion called the "invisible" bit, which is just one part of the  
Finder Info (which also contains things like file type and creator  
information, icon position, etc.).

Mac OS X has a mechanism for storing Finder Info on file systems  
without native support for Finder Info, by using extra files known as  
AppleDouble or "dot underscore" files (because their name is the  
original file's name with "._" prepended).  The management of the  
AppleDouble files happens in application-layer libraries in Mac OS X  
10.3.x and earlier, and also in the VFS layer in Mac OS X 10.4.x and
(Continue reading)

Permalink | Reply | 11 comments |
headers
Robert Watson | 1 Mar 2006 19:22
Picon
Favicon

Re: Add a file flag for "hidden" files?


On Wed, 1 Mar 2006, Mark Day wrote:

> I could certainly provide patches to FreeBSD for the headers, FAT, NTFS, and 
> probably HFS.  The strtofflags(3) and fflagstostr(3) functions should also 
> change so you can get at the flag bit via ls(1), find(1), chflags(1), etc.; 
> I think I could provide that patch, too.  (I've never actually used FreeBSD, 
> so it will take me a bit of time to get it installed, find my way around the 
> sources, make and test the changes.)

I don't see any problem with this.  The only flag I know of in FreeBSD that 
might not appear in Mac OS X is the system snapshot flag, which was added to 
UFS after Apple forked from the FreeBSD source.  You can find current flag 
allocation here:

   http://fxr.watson.org/fxr/source/sys/stat.h

274 /*
275  * Definitions of flags stored in file flags word.
276  *
277  * Super-user and owner changeable flags.
278  */
279 #define UF_SETTABLE     0x0000ffff      /* mask of owner changeable flags 
*/
280 #define UF_NODUMP       0x00000001      /* do not dump file */
281 #define UF_IMMUTABLE    0x00000002      /* file may not be changed */
282 #define UF_APPEND       0x00000004      /* writes to file may only append 
*/
283 #define UF_OPAQUE       0x00000008      /* directory is opaque wrt. union 
*/
(Continue reading)

Permalink | Reply | 5 comments |
headers
Eric Anderson | 1 Mar 2006 19:20
Favicon

Re: Add a file flag for "hidden" files?

Mark Day wrote:
> Hello,
>
> For an future version of Mac OS X, I'm trying to solve a problem where 
> certain files and directories on Windows file systems don't normally 
> appear in the Windows GUI, but currently do appear in the Mac OS X GUI 
> (but they shouldn't).  I'd like to gauge interest in having the change 
> ported back to FreeBSD (since many of our file systems are derived 
> from FreeBSD).  The approach I'm planning to use is to add a new flag 
> to the file flags (i.e. the st_flags field from stat(2); settable via 
> chflags(2)).  I'd like to avoid a collision with any flags FreeBSD 
> would add in the future, so either have FreeBSD adopt the flag, or 
> start allocating Mac OS X-specific flags from the high order bits 
> (since FreeBSD has been allocating them from the low order bits).  My 
> current name for this new file flag is UF_HIDDEN.
[..snip..]

> I could certainly provide patches to FreeBSD for the headers, FAT, 
> NTFS, and probably HFS.  The strtofflags(3) and fflagstostr(3) 
> functions should also change so you can get at the flag bit via ls(1), 
> find(1), chflags(1), etc.; I think I could provide that patch, too.  
> (I've never actually used FreeBSD, so it will take me a bit of time to 
> get it installed, find my way around the sources, make and test the 
> changes.)
>

Wow - about a week ago, I started looking into this same feature for 
FreeBSD, because I'd like to hide directories and files for UFS2 
snapshots.  I came to nearly the same conclusion, using chflags, etc, 
and even the name for the flag. :)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mark Day | 1 Mar 2006 19:32
Picon
Favicon

Re: Add a file flag for "hidden" files?

On Mar 1, 2006, at 10:22 AM, Robert Watson wrote:

> I don't see any problem with this.  The only flag I know of in  
> FreeBSD that might not appear in Mac OS X is the system snapshot  
> flag, which was added to UFS after Apple forked from the FreeBSD  
> source.

Mac OS X is missing both the NOUNLINK and the SNAPSHOT flags.

Since Mac OS X doesn't implement the functionality corresponding to  
either bit, my plan was to leave those #defines commented out in the  
Mac OS X headers, with a comment indicating we don't implement that  
functionality.  I'm a little concerned that an app might see those  
bits defined, try to use the functionality, and be surprised when it  
doesn't work as expected.

> We have talked about adding a flag to hint the presence of extended  
> ACL data also, so that applications know if they should rely solely  
> on stat() for protection information, or also call acl_get_ 
> {fd,file,link}() to receive extended ACL data for ls(1) output.  Is  
> your plan to mask hidden files solely in user space, or to look at  
> masking it in kernel also?

I knew I'd forgotten to mention something. The plan is that this bit  
is purely a hint to user space.  It's up to the application  
(especially a GUI) to decide whether to show a hidden file to the  
user or not.

-Mark
_______________________________________________
(Continue reading)

Permalink | Reply | 2 comments |
headers
Robert Watson | 1 Mar 2006 19:47
Picon
Favicon

Re: Add a file flag for "hidden" files?


On Wed, 1 Mar 2006, Mark Day wrote:

> On Mar 1, 2006, at 10:22 AM, Robert Watson wrote:
>
>> I don't see any problem with this.  The only flag I know of in FreeBSD that 
>> might not appear in Mac OS X is the system snapshot flag, which was added 
>> to UFS after Apple forked from the FreeBSD source.
>
> Mac OS X is missing both the NOUNLINK and the SNAPSHOT flags.
>
> Since Mac OS X doesn't implement the functionality corresponding to either 
> bit, my plan was to leave those #defines commented out in the Mac OS X 
> headers, with a comment indicating we don't implement that functionality. 
> I'm a little concerned that an app might see those bits defined, try to use 
> the functionality, and be surprised when it doesn't work as expected.

Sounds good -- as long as they are marked as reserved values in Mac OS X, that 
should (hopefully) prevent later collisions.

>> We have talked about adding a flag to hint the presence of extended ACL 
>> data also, so that applications know if they should rely solely on stat() 
>> for protection information, or also call acl_get_{fd,file,link}() to 
>> receive extended ACL data for ls(1) output.  Is your plan to mask hidden 
>> files solely in user space, or to look at masking it in kernel also?
>
> I knew I'd forgotten to mention something. The plan is that this bit is 
> purely a hint to user space.  It's up to the application (especially a GUI) 
> to decide whether to show a hidden file to the user or not.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Mark Day | 1 Mar 2006 21:24
Picon
Favicon

Re: Add a file flag for "hidden" files?

On Mar 1, 2006, at 10:47 AM, Robert Watson wrote:

>>> We have talked about adding a flag to hint the presence of  
>>> extended ACL data also, so that applications know if they should  
>>> rely solely on stat() for protection information, or also call  
>>> acl_get_{fd,file,link}() to receive extended ACL data for ls(1)  
>>> output.  Is your plan to mask hidden files solely in user space,  
>>> or to look at masking it in kernel also?
>>
>> I knew I'd forgotten to mention something. The plan is that this  
>> bit is purely a hint to user space.  It's up to the application  
>> (especially a GUI) to decide whether to show a hidden file to the  
>> user or not.
>
> Sounds good.  Do you plan to implement a hidden flag only in the  
> user flag range, or also a system hidden flag?

Just in the user flag range.

-Mark
_______________________________________________
freebsd-fs <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
Permalink | Reply | 0 comments |
headers
conrad | 2 Mar 2006 08:59
Picon

Re: Add a file flag for "hidden" files?

At 9:48 AM -0800 3/1/06, Mark Day wrote:

>For an future version of Mac OS X, I'm trying to solve a problem
>where certain files and directories on Windows file systems don't
>normally appear in the Windows GUI, but currently do appear in the
>Mac OS X GUI (but they shouldn't).  I'd like to gauge interest in
>having the change ported back to FreeBSD (since many of our file
>systems are derived from FreeBSD).  The approach I'm planning to use
>is to add a new flag to the file flags (i.e. the st_flags field from
>stat(2); settable via chflags(2)).  I'd like to avoid a collision
>with any flags FreeBSD would add in the future, so either have
>FreeBSD adopt the flag, or start allocating Mac OS X-specific flags
>from the high order bits (since FreeBSD has been allocating them
>from the low order bits).  My current name for this new file flag is
>UF_HIDDEN.
>
>Windows file systems (FAT, NTFS, SMB) have a "hidden" attribute bit
>on files and directories.  If a file or directory has this attribute
>set, Windows will not normally show it in the GUI unless you
>explicit ask to view hidden items.  Mac file systems (HFS, AFP) have
>a similar notion called the "invisible" bit, which is just one part
>of the Finder Info (which also contains things like file type and
>creator information, icon position, etc.).
>
>Mac OS X has a mechanism for storing Finder Info on file systems
>without native support for Finder Info, by using extra files known
>as AppleDouble or "dot underscore" files (because their name is the
>original file's name with "._" prepended).  The management of the
>AppleDouble files happens in application-layer libraries in Mac OS X
>10.3.x and earlier, and also in the VFS layer in Mac OS X 10.4.x and
(Continue reading)

Permalink | Reply | 1 comment |
headers
Mark Day | 2 Mar 2006 18:35
Picon
Favicon

Re: Add a file flag for "hidden" files?

Hi Conrad,

On Mar 1, 2006, at 11:59 PM, conrad <at> mac.com wrote:

> This is great with MS' SMB servers, and also with local  
> filesystems, as they can easily have UF_HIDDEN added.

Yeah, George is eagerly awaiting the bit so he can use it in SMB.

>   What are you thinking for other filesystems, for remotely served  
> filesystems, that is, which for whatever reason don't read/write  
> UF_HIDDEN?

If they can pass the bit through and have it preserved somewhere,  
then do that.  (If the server doesn't understand the meaning of the  
bit, then at least preserving it for clients seems better than  
ignoring it or returning an error.)

Otherwise, I suppose they'd have to treat it like any other flag bit  
that they don't understand, or the server doesn't support.  I assume  
that means returning an error if the caller attempts to set  
UF_HIDDEN.  Looking at the error codes listed for chflags(2) [in the  
online FreeBSD man page], there is no EINVAL listed, but there is  
EPERM and EOPNOTSUPP.  I'm not sure which one I'd choose.

I can also see an argument that such a file system should just ignore  
the UF_HIDDEN bit and not return an error because the flag is meant  
as a hint to applications, not meant to be enforced by the file system.

FYI, Mac OS X has a mechanism for a file system to tell its clients
(Continue reading)

Permalink | Reply | 0 comments |
headers
Mark Day | 2 Mar 2006 18:45
Picon
Favicon

Re: Add a file flag for "hidden" files?

On Mar 1, 2006, at 10:22 AM, Robert Watson wrote:

> You can find current flag allocation here:
>
>   http://fxr.watson.org/fxr/source/sys/stat.h

The obvious (to me) choice for UF_HIDDEN would be 0x00000040 (leaving  
0x00000020 unused as the user flag corresponding to SF_SNAPSHOT).   
But would anyone object to using 0x00008000 (the most significant bit  
of the user flags)?

The reason I ask is because HFS Plus only stores 16 bits of file  
flags on disk (lower 8 bits of user flags plus lower 8 bits of super- 
user flags).  We're getting close to running out of the lower 8 bits,  
and HFS Plus has a different place it can store UF_HIDDEN (the  
"invisible" bit of the Finder Info).  There's a side bonus, too,  
because the UF_HIDDEN and "invisible" bits can't get out of sync if  
the UF_HIDDEN flag isn't actually being stored on disk.  In the Mac  
OS X sources, I already have code in HFS to propagate the "invisible"  
bit to UF_HIDDEN and vice versa to make sure they always appear  
consistent (if either one is set, then both are set).

Or would you prefer I use 0x00000020 since there is no user flag  
equivalent of SF_SNAPSHOT?

-Mark

_______________________________________________
freebsd-fs <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
(Continue reading)

Permalink | Reply | 1 comment |
headers
Robert Watson | 3 Mar 2006 01:24
Picon
Favicon

Re: Add a file flag for "hidden" files?

On Thu, 2 Mar 2006, Mark Day wrote:

>> You can find current flag allocation here:
>> 
>>   http://fxr.watson.org/fxr/source/sys/stat.h
>
> The obvious (to me) choice for UF_HIDDEN would be 0x00000040 (leaving 
> 0x00000020 unused as the user flag corresponding to SF_SNAPSHOT).  But would 
> anyone object to using 0x00008000 (the most significant bit of the user 
> flags)?

I see no reason not to do this -- I also checked in NetBSD and OpenBSD source, 
and neither has allocated any additional flags (NetBSD has picked up the 
snapshot flag from FreeBSD, but OpenBSD has not).

Robert N M Watson
_______________________________________________
freebsd-fs <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe <at> freebsd.org"
Permalink | Reply | 0 comments |

Gmane